From 8723fa59c582ff8d0c9a9760b7cb4d28ef7f986c Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Wed, 9 Jul 2025 15:40:52 -0500 Subject: [PATCH 01/16] Update Spectator-CPP-2.0 Docs --- docs/spectator/lang/cpp/meters/age-gauge.md | 39 +++ docs/spectator/lang/cpp/meters/counter.md | 18 + .../spectator/lang/cpp/meters/dist-summary.md | 13 + docs/spectator/lang/cpp/meters/gauge.md | 21 ++ docs/spectator/lang/cpp/meters/max-gauge.md | 10 + .../lang/cpp/meters/monotonic-counter-uint.md | 11 + .../lang/cpp/meters/monotonic-counter.md | 11 + .../cpp/meters/percentile-dist-summary.md | 15 + .../lang/cpp/meters/percentile-timer.md | 24 ++ docs/spectator/lang/cpp/meters/timer.md | 16 + docs/spectator/lang/cpp/migrations.md | 0 docs/spectator/lang/cpp/perf-test.md | 0 docs/spectator/lang/cpp/usage.md | 314 +++++++++++++----- 13 files changed, 408 insertions(+), 84 deletions(-) create mode 100644 docs/spectator/lang/cpp/meters/age-gauge.md create mode 100644 docs/spectator/lang/cpp/meters/counter.md create mode 100644 docs/spectator/lang/cpp/meters/dist-summary.md create mode 100644 docs/spectator/lang/cpp/meters/gauge.md create mode 100644 docs/spectator/lang/cpp/meters/max-gauge.md create mode 100644 docs/spectator/lang/cpp/meters/monotonic-counter-uint.md create mode 100644 docs/spectator/lang/cpp/meters/monotonic-counter.md create mode 100644 docs/spectator/lang/cpp/meters/percentile-dist-summary.md create mode 100644 docs/spectator/lang/cpp/meters/percentile-timer.md create mode 100644 docs/spectator/lang/cpp/meters/timer.md create mode 100644 docs/spectator/lang/cpp/migrations.md create mode 100644 docs/spectator/lang/cpp/perf-test.md diff --git a/docs/spectator/lang/cpp/meters/age-gauge.md b/docs/spectator/lang/cpp/meters/age-gauge.md new file mode 100644 index 000000000..84eb4e5a2 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/age-gauge.md @@ -0,0 +1,39 @@ +The value is the time in seconds since the epoch at which an event has successfully occurred, or +`0` to use the current time in epoch seconds. After an Age Gauge has been set, it will continue +reporting the number of seconds since the last time recorded, for as long as the SpectatorD +process runs. The purpose of this metric type is to enable users to more easily implement the +Time Since Last Success alerting pattern. + +To set a specific time as the last success: + +```cpp + +``` + +To set `now()` as the last success: + +```cpp + +``` + +By default, a maximum of `1000` Age Gauges are allowed per `spectatord` process, because there is no +mechanism for cleaning them up. This value may be tuned with the `--age_gauge_limit` flag on the +`spectatord` binary. + +Since Age Gauges are long-lived entities that reside in the memory of the SpectatorD process, if +you need to delete and re-create them for any reason, then you can use the [SpectatorD admin server] +to accomplish this task. You can delete all Age Gauges or a single Age Gauge. + +**Example:** + +``` +curl -X DELETE \ +http://localhost:1234/metrics/A +``` + +``` +curl -X DELETE \ +http://localhost:1234/metrics/A/fooIsTheName,some.tag=val1,some.otherTag=val2 +``` + +[SpectatorD admin server]: ../../../agent/usage.md#admin-server diff --git a/docs/spectator/lang/cpp/meters/counter.md b/docs/spectator/lang/cpp/meters/counter.md new file mode 100644 index 000000000..bb6942b83 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/counter.md @@ -0,0 +1,18 @@ +A Counter is used to measure the rate at which an event is occurring. Considering an API endpoint, +a Counter could be used to measure the rate at which it is being accessed. + +Counters are reported to the backend as a rate-per-second. In Atlas, the `:per-step` operator can +be used to convert them back into a value-per-step on a graph. + +Call `increment()` when an event occurs: + +```cpp + +``` + +You can also pass a value to `increment()`. This is useful when a collection of events happens +together: + +```cpp + +``` diff --git a/docs/spectator/lang/cpp/meters/dist-summary.md b/docs/spectator/lang/cpp/meters/dist-summary.md new file mode 100644 index 000000000..c44ec8be0 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/dist-summary.md @@ -0,0 +1,13 @@ +A Distribution Summary is used to track the distribution of events. It is similar to a Timer, but +more general, in that the size does not have to be a period of time. For example, a Distribution +Summary could be used to measure the payload sizes of requests hitting a server. + +Always use base units when recording data, to ensure that the tick labels presented on Atlas graphs +are readable. If you are measuring payload size, then use bytes, not kilobytes (or some other unit). +This means that a `4K` tick label will represent 4 kilobytes, rather than 4 kilo-kilobytes. + +Call `record()` with a value: + +```cpp + +``` diff --git a/docs/spectator/lang/cpp/meters/gauge.md b/docs/spectator/lang/cpp/meters/gauge.md new file mode 100644 index 000000000..777de1157 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/gauge.md @@ -0,0 +1,21 @@ +A gauge is a value that is sampled at some point in time. Typical examples for gauges would be +the size of a queue or number of threads in a running state. Since gauges are not updated inline +when a state change occurs, there is no information about what might have occurred between samples. + +Consider monitoring the behavior of a queue of tasks. If the data is being collected once a minute, +then a gauge for the size will show the size when it was sampled. The size may have been much +higher or lower at some point during interval, but that is not known. + +Call `set()` with a value: + +```cpp + +``` + +Gauges will report the last set value for 15 minutes. This done so that updates to the values do +not need to be collected on a tight 1-minute schedule to ensure that Atlas shows unbroken lines in +graphs. A custom TTL may be configured for gauges. SpectatorD enforces a minimum TTL of 5 seconds. + +```cpp + +``` diff --git a/docs/spectator/lang/cpp/meters/max-gauge.md b/docs/spectator/lang/cpp/meters/max-gauge.md new file mode 100644 index 000000000..6760ff44e --- /dev/null +++ b/docs/spectator/lang/cpp/meters/max-gauge.md @@ -0,0 +1,10 @@ +The value is a number that is sampled at a point in time, but it is reported as a maximum Gauge +value to the backend. This ensures that only the maximum value observed during a reporting interval +is sent to the backend, thus over-riding the last-write-wins semantics of standard Gauges. Unlike +standard Gauges, Max Gauges do not continue to report to the backend, and there is no TTL. + +Call `set()` with a value: + +```cpp + +``` diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md new file mode 100644 index 000000000..5f13622f8 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md @@ -0,0 +1,11 @@ +A Monotonic Counter (uint64) is used to measure the rate at which an event is occurring, when the +source data is a monotonically increasing number. A minimum of two samples must be sent, in order to +calculate a delta value and report it to the backend as a rate-per-second. A variety of networking +metrics may be reported monotonically, and this metric type provides a convenient means of recording +these values, at the expense of a slower time-to-first metric. + +Call `set()` when an event occurs: + +```cpp + +``` diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter.md b/docs/spectator/lang/cpp/meters/monotonic-counter.md new file mode 100644 index 000000000..991dbdab3 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/monotonic-counter.md @@ -0,0 +1,11 @@ +A Monotonic Counter (float) is used to measure the rate at which an event is occurring, when the +source data is a monotonically increasing number. A minimum of two samples must be sent, in order to +calculate a delta value and report it to the backend as a rate-per-second. A variety of networking +metrics may be reported monotonically, and this metric type provides a convenient means of recording +these values, at the expense of a slower time-to-first metric. + +Call `set()` when an event occurs: + +```cpp + +``` diff --git a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md new file mode 100644 index 000000000..bb65a24b4 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md @@ -0,0 +1,15 @@ +The value tracks the distribution of events, with percentile estimates. It is similar to a +`PercentileTimer`, but more general, because the size does not have to be a period of time. + +For example, it can be used to measure the payload sizes of requests hitting a server or the +number of records returned from a query. + +In order to maintain the data distribution, they have a higher storage cost, with a worst-case of +up to 300X that of a standard Distribution Summary. Be diligent about any additional dimensions +added to Percentile Distribution Summaries and ensure that they have a small bounded cardinality. + +Call `record()` with a value: + +```cpp + +``` diff --git a/docs/spectator/lang/cpp/meters/percentile-timer.md b/docs/spectator/lang/cpp/meters/percentile-timer.md new file mode 100644 index 000000000..594f755c1 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/percentile-timer.md @@ -0,0 +1,24 @@ +The value is the number of seconds that have elapsed for an event, with percentile estimates. + +This metric type will track the data distribution by maintaining a set of Counters. The +distribution can then be used on the server side to estimate percentiles, while still +allowing for arbitrary slicing and dicing based on dimensions. + +In order to maintain the data distribution, they have a higher storage cost, with a worst-case of +up to 300X that of a standard Timer. Be diligent about any additional dimensions added to Percentile +Timers and ensure that they have a small bounded cardinality. + +Call `record()` with a value: + +```cpp + +``` + +A `StopWatch` class is available, which may be used as a [Context Manager] to automatically record +the number of seconds that have elapsed while executing a block of code: + +```cpp + +``` + +[Context Manager]: https://docs.python.org/3/reference/datamodel.html#context-managers diff --git a/docs/spectator/lang/cpp/meters/timer.md b/docs/spectator/lang/cpp/meters/timer.md new file mode 100644 index 000000000..1fe9e8675 --- /dev/null +++ b/docs/spectator/lang/cpp/meters/timer.md @@ -0,0 +1,16 @@ +A Timer is used to measure how long (in seconds) some event is taking. + +Call `record()` with a value: + +```cpp + +``` + +A `StopWatch` class is available, which may be used as a [Context Manager] to automatically record +the number of seconds that have elapsed while executing a block of code: + +```cpp + +``` + +[Context Manager]: https://docs.python.org/3/reference/datamodel.html#context-managers diff --git a/docs/spectator/lang/cpp/migrations.md b/docs/spectator/lang/cpp/migrations.md new file mode 100644 index 000000000..e69de29bb diff --git a/docs/spectator/lang/cpp/perf-test.md b/docs/spectator/lang/cpp/perf-test.md new file mode 100644 index 000000000..e69de29bb diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index b15e3e2be..71d935273 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -1,112 +1,258 @@ # spectator-cpp Usage -C++ thin-client [metrics library] for use with [Atlas] and [SpectatorD]. +CPP thin-client [metrics library] for use with [Atlas] and [SpectatorD]. [metrics library]: https://github.com/Netflix/spectator-cpp [Atlas]: ../../../overview.md [SpectatorD]: ../../agent/usage.md +## Supported CPP Versions + +This library currently utilzes C++ 20. + +## Installing & Building + +If your project utilizes CMake you can incorporate this project by simply calling add_subdirectory(spectator-cpp). +If you wish to simply build the Spectator-CPP thin client you can utilize the Docker containter instructions found +here https://github.com/Netflix/spectator-cpp/tree/main/Dockerfiles. The container installs a minimal set of depencies +to build the project in the container, such as g++-13, python3, and conan. The project only has 3 external dependencies +spdlog, gtest and boost. These dependencies are managed through conan. + ## Instrumenting Code -```C++ -#include - -// use default values -static constexpr auto kDefault = 0; - -struct Request { - std::string country; -}; - -struct Response { - int status; - int size; -}; - -class Server { - public: - explicit Server(spectator::Registry* registry) - : registry_{registry}, - request_count_id_{registry->CreateId("server.requestCount", spectator::Tags{})}, - request_latency_{registry->GetTimer("server.requestLatency")}, - response_size_{registry->GetDistributionSummary("server.responseSizes")} {} - - Response Handle(const Request& request) { - auto start = std::chrono::steady_clock::now(); - - // do some work and obtain a response... - Response res{200, 64}; - - // Update the Counter id with dimensions, based on information in the request. The Counter - // will be looked up in the Registry, which is a fairly cheap operation, about the same as - // the lookup of an id object in a map. However, it is more expensive than having a local - // variable set to the Counter. - auto cnt_id = request_count_id_ - ->WithTag("country", request.country) - ->WithTag("status", std::to_string(res.status)); - registry_->GetCounter(std::move(cnt_id))->Increment(); - request_latency_->Record(std::chrono::steady_clock::now() - start); - response_size_->Record(res.size); - return res; - } +```cpp +#include + +int main() +{ + // Create common tags to be applied to all metrics sent to Atlas + std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; + + // Create a config which defines the way you send metrics to SpectatorD + auto config = Config(WriterConfig(WriterTypes::Memory), commonTags); + + // Initialize the Registry with the Config (Always required before sending metrics) + auto r = Registry(config); + + // Create some meters + auto threadGauge = r.gauge("threads"); + auto queueGauge = r.gauge("queue-size", {{"my-tags", "bar"}}); + + auto memoryWriter = static_cast(WriterTestHelper::GetImpl()); + EXPECT_TRUE(memoryWriter->IsEmpty()); + + threadGauge.Set(GetNumThreads()); // Metric sent to SpectatorD: "g:threads,platform=my-platform,process=my-process:5.000000\n" + queueGauge.Set(GetQueueSize()); // Metric sent to SpectatorD: "g:queue-size,my-tags=bar,platform=my-platform,process=my-process:10.000000\n" +} +``` + +## Logging + +Logging is implemented with spdlog and the default location is standard output. The default log level is spdlog::level::info. The Logger class is a singleton +and provides a function Logger::GetLogger(). You can change the logger level by simply calling Logger::GetLogger()->set_level(spdlog::level::debug); as long as +the logger has been created successfully + +## Runtime Metrics - private: - spectator::Registry* registry_; - std::shared_ptr request_count_id_; - std::shared_ptr request_latency_; - std::shared_ptr response_size_; -}; +Coming Soon -Request get_next_request() { - return Request{"US"}; + +## Working with MeterId Objects + +Each metric stored in Atlas is uniquely identified by the combination of the name and the tags +associated with it. In `spectator-cpp`, this data is represented with `Id` objects, created +by the `Registry`. The `NewId()` method returns new `Id` objects, which have extra common +tags applied, and which can be further customized by calling the `WithTag()` and `WithTags()` +methods. Each `Id` will create and store a validated subset of the `spectatord` protocol line +to be written for each `Meter`, when it is instantiated. `Id` objects can be passed around and +used concurrently. Manipulating the tags with the provided methods will create new `Id` objects. + +Note that **all tag keys and values must be strings.** For example, if you want to keep track of the +number of successful requests, then you must cast integers to strings. The `Id` class will +validate these values, dropping or changing any that are not valid, and reporting a warning log. + +```cpp +#include + +int main() +{ + std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; + + auto config = Config(WriterConfig(WriterTypes::Memory), commonTags); + auto registry = Registry(config); + + registry.counter("server.requests", {{"statusCode", std::to_string(200)}}).Increment(); + + // Option 1: Using the registry to create a MeterId + auto numRequestsId = registry.new_id("server.numRequests", {{"statusCode", std::to_string(200)}}); + // Creating the counter with the MeterId and Incrementing it + registry.counter_with_id(numRequestsId).Increment(); + + // Option 2: Directly creating a Counter + auto numRequestsCounter = registry.counter("server.numRequests", {{"statusCode", std::to_string(200)}}); + numRequestsCounter.Increment(); } +``` + +Atlas metrics will be consumed by users many times after the data has been reported, so they should +be chosen thoughtfully, while considering how they will be used. See the [naming conventions] page +for general guidelines on metrics naming and restrictions. + +[naming conventions]: ../../../concepts/naming.md + +## Meter Types + +* [Age Gauge](./meters/age-gauge.md) +* [Counter](./meters/counter.md) +* [Distribution Summary](./meters/dist-summary.md) +* [Gauge](./meters/gauge.md) +* [Max Gauge](./meters/max-gauge.md) +* [Monotonic Counter](./meters/monotonic-counter.md) +* [Monotonic Counter Uint](./meters/monotonic-counter-uint.md) +* [Percentile Distribution Summary](./meters/percentile-dist-summary.md) +* [Percentile Timer](./meters/percentile-timer.md) +* [Timer](./meters/timer.md) + +## Output Location -int main() { - auto logger = spdlog::stdout_color_mt("console"); - std::unordered_map common_tags{{"xatlas.process", "some-sidecar"}}; - spectator::Config cfg{"unix:/run/spectatord/spectatord.unix", common_tags}; - spectator::Registry registry{std::move(cfg), logger); +`spectator.Registry` now supports three different writers. The writer types are Memory Writer, UDP Writer, +and a UDS (Unix Domain Socket) Writer - Server server{®istry}; +Possible values are: - for (auto i = 1; i <= 3; ++i) { - // get a request - auto req = get_next_request(); - server.Handle(req); +* `""` - Empty string will default to `udp`, with the `LineBuffer` disabled by default. +* `none` - A no-op writer that does nothing. Used to disable metrics collection. +* `memory` - Write to memory. Useful for testing. +* `stderr` - Write to standard error for the process. +* `stdout` - Write to standard output for the process. +* `udp` - Write to the default UDP port for `spectatord`. This is the default location. +* `unix` - Write to the default Unix Domain Socket for `spectatord`. Useful for high-volume scenarios. +* `file:///path/to/file` - Write to a custom file (e.g. `file:///tmp/foo/bar`). +* `udp://host:port` - Write to a custom UDP socket (e.g. `udp://127.0.0.1:1235`). +* `unix:///path/to/socket` - Write to a custom Unix domain socket (e.g. `unix:///tmp/some.socket`). + +Location can also be set through the environment variable `SPECTATOR_OUTPUT_LOCATION`. If both are set, +the environment variable takes precedence over the passed config. If either values provided to the WriterConfig are +invalid a runtime exception will be thrown. + +## Line Buffer + +The `NewConfigWithBuffer` factory function takes a `bufferSize` parameter that configures an optional +`LineBuffer`, which caches protocol lines locally, before flushing them to `spectatord`. Flushes occur +under two conditions: (1) the buffer size is exceeded, or (2) five seconds has elapsed. The buffer is +available for the `UdpWriter` and the `UnixgramWriter`, where performance matters most. The `LineBuffer` +is disabled by default (with size zero) in the standard `NewConfig` factory function, to ensure that the +default operation of the library works under most circumstances. For high-performance scenarios, a 60KB +buffer size is recommended. The maximum buffer size for udp sockets and unix domain sockets on Linux is +64KB, so stay under this limit. + +## Batch Usage + +When using `spectator-go` to report metrics from a batch job, ensure that the batch job runs for at +least five (5), if not ten (10) seconds in duration. This is necessary in order to allow sufficient +time for `spectatord` to publish metrics to the Atlas backend; it publishes every five seconds. If +your job does not run this long, or you find you are missing metrics that were reported at the end +of your job run, then add a five-second sleep before exiting. This will allow time for the metrics +to be sent. + +## Debug Metrics Delivery to `spectatord` + +In order to see debug log messages from `spectatord`, create an `/etc/default/spectatord` file with +the following contents: + +```shell +SPECTATORD_OPTIONS="--verbose" +``` + +This will report all metrics that are sent to the Atlas backend in the `spectatord` logs, which will +provide an opportunity to correlate metrics publishing events from your client code. + +## Design Considerations - Reporting Intervals + +This client is stateless, and sends a UDP packet (or unixgram) to `spectatord` each time a meter is +updated. If you are performing high-volume operations, on the order of tens-of-thousands or millions +of operations per second, then you should pre-aggregate your metrics and report them at a cadence +closer to the `spectatord` publish interval of 5 seconds. This will keep the CPU usage related to +`spectator-go` and `spectatord` low (around 1% or less), as compared to up to 40% for high-volume +scenarios. + +## Writing Tests + +To write tests against this library, instantiate a test instance of the `Registry` and configure it +to use the [MemoryWriter](https://github.com/Netflix/spectator-go/blob/main/spectator/writer/writer.go#L18-L21), +which stores all updates in an `Array`. Maintain a handle to the `MemoryWriter`, then inspect the +`Lines()` to verify your metrics updates. See the source code for more testing examples. + +```golang +import ( + "fmt" + "github.com/Netflix/spectator-go/v2/spectator" + "github.com/Netflix/spectator-go/v2/spectator/writer" + "testing" + "time" +) + +func TestRegistryWithMemoryWriter_Counter(t *testing.T) { + config, _ := spectator.NewConfig("memory", nil, nil) + registry, _ = spectator.NewRegistry(config) + mw := registry.GetWriter().(*writer.MemoryWriter) + + counter := registry.Counter("test_counter", nil) + counter.Increment() + + expected := "c:test_counter:1" + if len(mw.Lines()) != 1 || mw.Lines()[0] != expected { + t.Errorf("Expected '%s', got '%s'", expected, mw.Lines()[0]) } } ``` -## Usage +### Protocol Parser + +A [SpectatorD] line protocol parser is available, which can be used for validating the results +captured by a `MemoryWriter`. + +```golang +import ( + "github.com/Netflix/spectator-go/v2/spectator" + "testing" +) + +func TestParseProtocolLineWithValidInput(t *testing.T) { + line := "c:name,tag1=value1,tag2=value2:50" + meterType, meterId, value, err := spectator.ParseProtocolLine(line) + + if err != nil { + t.Errorf("Unexpected error: %v", err) + } -We do not publish this library as a binary artifact, because it can be used across a variety of CPU -and OS platforms, and we do not want to incur this support overheard for a library that is not on -the Paved Path. However, this is a Conan 2 and CMake project, so you can pull the latest code, and -add some build configuration to use it in your project. + if meterType != "c" { + t.Errorf("Unexpected meter type: %v", meterType) + } -As an example of how this is done, see the [atlas-system-agent] project. + if meterId.Name() != "name" || meterId.Tags()["tag1"] != "value1" || meterId.Tags()["tag2"] != "value2" { + t.Errorf("Unexpected meter id: %v", meterId) + } -* Download the latest `spectator-cpp` code ([conanfile.py#L39-L57]). -* Add the library to your CMake build ([lib/CMakeLists.txt#L1-L32]). + if value != "50" { + t.Errorf("Unexpected value: %v", value) + } +} +``` -This library has a few dependencies ([conanfile.py#L6-L13]), including a recent `abseil`. +[SpectatorD]: ../../agent/usage.md -[atlas-system-agent]: https://github.com/Netflix-Skunkworks/atlas-system-agent/tree/main -[conanfile.py#L39-L57]: https://github.com/Netflix-Skunkworks/atlas-system-agent/blob/main/conanfile.py#L39-L57 -[lib/CMakeLists.txt#L1-L32]: https://github.com/Netflix-Skunkworks/atlas-system-agent/blob/main/lib/CMakeLists.txt#L1-L32 -[conanfile.py#L6-L13]: https://github.com/Netflix/spectator-cpp/blob/main/conanfile.py#L6-L13 +## Performance -### High-Volume Publishing +On an `m5d.2xlarge` EC2 instance, with `Go 1.24.3` and `github.com/Netflix/spectator-go/v2 v2.0.13`, we +have observed the following single-threaded performance numbers across a two-minute test window: -By default, the library sends every meter change to the `spectatord` sidecar immediately. This -involves a blocking `send` call and underlying system calls, and may not be the most efficient way -to publish metrics in high-volume use cases. +* 135,771 requests/second over `udp` +* 206,641 requests/second over `unix` -For this purpose, a simple buffering functionality in `Publisher` is implemented, and it can be -turned on by passing a buffer size to the `spectator::Config` constructor ([config.h#L8-L12]). It -is important to note that, until this buffer fills up, the `Publisher` will not send any meters to -the sidecar. Therefore, if your application doesn't emit meters at a high rate, you should either -keep the buffer very small, or do not configure a buffer size at all, which will fall back to the -"publish immediately" mode of operation. +The benchmark incremented a single counter with two tags in a tight loop, to simulate real-world tag +usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol line +was `74` characters in length. -[config.h#L8-L12]: https://github.com/Netflix/spectator-cpp/blob/main/spectator/config.h#L8-L12 +The Go process CPU usage was ~112% and the `spectatord` process CPU usage was ~62% on this 8 vCPU +system, for `udp`. It was ~113% and ~85%, respectively, for `unix`. From 504e6d744dca5e49b3127a7f24269ac6473462a8 Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Wed, 9 Jul 2025 15:54:49 -0500 Subject: [PATCH 02/16] add performance test --- docs/spectator/lang/cpp/perf-test.md | 63 ++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) diff --git a/docs/spectator/lang/cpp/perf-test.md b/docs/spectator/lang/cpp/perf-test.md index e69de29bb..e9b01022c 100644 --- a/docs/spectator/lang/cpp/perf-test.md +++ b/docs/spectator/lang/cpp/perf-test.md @@ -0,0 +1,63 @@ +# Performance + +## Test Script + +Test maximum single-threaded throughput for two minutes. + +```cpp +#include +#include +#include +#include +#include +#include + + +int main() +{ + Logger::info("Starting UDP performance test..."); + + auto r = Registry(Config(WriterConfig(WriterTypes::UDP))); + //auto r = Registry(Config(WriterConfig(WriterTypes::Unix))); + std::unordered_map tags = { {"location", "udp"}, {"version", "correct-horse-battery-staple"}}; + + // Set maximum duration to 2 minutes + constexpr int max_duration_seconds = 2 * 60; + + // Track iterations and timing + unsigned long long iterations = 0; + auto start_time = std::chrono::steady_clock::now(); + + // Helper function to get elapsed time in seconds + auto elapsed = [&start_time]() -> double { + auto now = std::chrono::steady_clock::now(); + return std::chrono::duration(now - start_time).count(); + }; + + while (true) + { + r.counter("udp_test_counter", tags).Increment(); + iterations++; + + if (iterations % 500000 == 0) + { + if (elapsed() > max_duration_seconds) + { + break; + } + } + } + + double total_elapsed = elapsed(); + double rate_per_second = iterations / total_elapsed; + + Logger::info("Iterations completed: {}", iterations); + Logger::info("Total elapsed time: {:.2f} seconds", total_elapsed); + Logger::info("Rate: {:.2f} iterations/second", rate_per_second); + return 0; +} +``` + +## Results + +See [Usage > Performance](usage.md#performance). \ No newline at end of file From cc11794e75618f7ce58c4c73c8362543493287e1 Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Wed, 9 Jul 2025 16:19:09 -0500 Subject: [PATCH 03/16] Add performance test to docs --- docs/spectator/lang/cpp/perf-test.md | 1 + docs/spectator/lang/cpp/usage.md | 60 ++++------------------------ mkdocs.yml | 15 ++++++- 3 files changed, 23 insertions(+), 53 deletions(-) diff --git a/docs/spectator/lang/cpp/perf-test.md b/docs/spectator/lang/cpp/perf-test.md index e9b01022c..6e258d259 100644 --- a/docs/spectator/lang/cpp/perf-test.md +++ b/docs/spectator/lang/cpp/perf-test.md @@ -19,6 +19,7 @@ int main() auto r = Registry(Config(WriterConfig(WriterTypes::UDP))); //auto r = Registry(Config(WriterConfig(WriterTypes::Unix))); + std::unordered_map tags = { {"location", "udp"}, {"version", "correct-horse-battery-staple"}}; // Set maximum duration to 2 minutes diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index 71d935273..db8e6ac18 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -20,6 +20,8 @@ spdlog, gtest and boost. These dependencies are managed through conan. ## Instrumenting Code +{% raw %} + ```cpp #include @@ -46,6 +48,8 @@ int main() } ``` +{% endraw %} + ## Logging Logging is implemented with spdlog and the default location is standard output. The default log level is spdlog::level::info. The Logger class is a singleton @@ -71,6 +75,8 @@ Note that **all tag keys and values must be strings.** For example, if you want number of successful requests, then you must cast integers to strings. The `Id` class will validate these values, dropping or changing any that are not valid, and reporting a warning log. +{% raw %} + ```cpp #include @@ -94,6 +100,8 @@ int main() } ``` +{% endraw %} + Atlas metrics will be consumed by users many times after the data has been reported, so they should be chosen thoughtfully, while considering how they will be used. See the [naming conventions] page for general guidelines on metrics naming and restrictions. @@ -183,63 +191,11 @@ to use the [MemoryWriter](https://github.com/Netflix/spectator-go/blob/main/spec which stores all updates in an `Array`. Maintain a handle to the `MemoryWriter`, then inspect the `Lines()` to verify your metrics updates. See the source code for more testing examples. -```golang -import ( - "fmt" - "github.com/Netflix/spectator-go/v2/spectator" - "github.com/Netflix/spectator-go/v2/spectator/writer" - "testing" - "time" -) - -func TestRegistryWithMemoryWriter_Counter(t *testing.T) { - config, _ := spectator.NewConfig("memory", nil, nil) - registry, _ = spectator.NewRegistry(config) - mw := registry.GetWriter().(*writer.MemoryWriter) - - counter := registry.Counter("test_counter", nil) - counter.Increment() - - expected := "c:test_counter:1" - if len(mw.Lines()) != 1 || mw.Lines()[0] != expected { - t.Errorf("Expected '%s', got '%s'", expected, mw.Lines()[0]) - } -} -``` - ### Protocol Parser A [SpectatorD] line protocol parser is available, which can be used for validating the results captured by a `MemoryWriter`. -```golang -import ( - "github.com/Netflix/spectator-go/v2/spectator" - "testing" -) - -func TestParseProtocolLineWithValidInput(t *testing.T) { - line := "c:name,tag1=value1,tag2=value2:50" - meterType, meterId, value, err := spectator.ParseProtocolLine(line) - - if err != nil { - t.Errorf("Unexpected error: %v", err) - } - - if meterType != "c" { - t.Errorf("Unexpected meter type: %v", meterType) - } - - if meterId.Name() != "name" || meterId.Tags()["tag1"] != "value1" || meterId.Tags()["tag2"] != "value2" { - t.Errorf("Unexpected meter id: %v", meterId) - } - - if value != "50" { - t.Errorf("Unexpected value: %v", value) - } -} -``` - [SpectatorD]: ../../agent/usage.md ## Performance diff --git a/mkdocs.yml b/mkdocs.yml index bed46ede0..c1b9bba66 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -228,6 +228,19 @@ nav: - Overview: spectator/lang/overview.md - C++: - Usage: spectator/lang/cpp/usage.md + - Meters: + - Age Gauge: spectator/lang/cpp/meters/age-gauge.md + - Counter: spectator/lang/cpp/meters/counter.md + - Distribution Summary: spectator/lang/cpp/meters/dist-summary.md + - Gauges: spectator/lang/cpp/meters/gauge.md + - Max Gauge: spectator/lang/cpp/meters/max-gauge.md + - Monotonic Counter: spectator/lang/cpp/meters/monotonic-counter.md + - Monotonic Counter Uint: spectator/lang/cpp/meters/monotonic-counter-uint.md + - Percentile Distribution Summary: spectator/lang/cpp/meters/percentile-dist-summary.md + - Percentile Timer: spectator/lang/cpp/meters/percentile-timer.md + - Timer: spectator/lang/cpp/meters/timer.md + - Migrations: spectator/lang/cpp/migrations.md + - Performance: spectator/lang/cpp/perf-test.md - Go: - Usage: spectator/lang/go/usage.md - Meters: @@ -235,7 +248,7 @@ nav: - Counter: spectator/lang/go/meters/counter.md - Distribution Summary: spectator/lang/go/meters/dist-summary.md - Gauges: spectator/lang/go/meters/gauge.md - - Max Gauge: spectator/lang/py/meters/max-gauge.md + - Max Gauge: spectator/lang/go/meters/max-gauge.md - Monotonic Counter: spectator/lang/go/meters/monotonic-counter.md - Monotonic Counter Uint: spectator/lang/go/meters/monotonic-counter-uint.md - Percentile Distribution Summary: spectator/lang/go/meters/percentile-dist-summary.md From 991f224bc6d2d4bc27654babf02fac341630aeee Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Thu, 10 Jul 2025 12:56:14 -0500 Subject: [PATCH 04/16] add meters --- docs/spectator/lang/cpp/meters/age-gauge.md | 30 ++++++++++++++++- docs/spectator/lang/cpp/meters/counter.md | 32 +++++++++++++++++-- .../spectator/lang/cpp/meters/dist-summary.md | 16 +++++++++- docs/spectator/lang/cpp/meters/gauge.md | 31 +++++++++++++++++- docs/spectator/lang/cpp/meters/max-gauge.md | 16 +++++++++- .../lang/cpp/meters/monotonic-counter-uint.md | 16 +++++++++- .../lang/cpp/meters/monotonic-counter.md | 16 +++++++++- .../cpp/meters/percentile-dist-summary.md | 16 +++++++++- .../lang/cpp/meters/percentile-timer.md | 29 ++++++++++------- docs/spectator/lang/cpp/meters/timer.md | 21 +++++++----- 10 files changed, 194 insertions(+), 29 deletions(-) diff --git a/docs/spectator/lang/cpp/meters/age-gauge.md b/docs/spectator/lang/cpp/meters/age-gauge.md index 84eb4e5a2..9d5c850df 100644 --- a/docs/spectator/lang/cpp/meters/age-gauge.md +++ b/docs/spectator/lang/cpp/meters/age-gauge.md @@ -7,13 +7,41 @@ Time Since Last Success alerting pattern. To set a specific time as the last success: ```cpp +#include +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create an Age Gauge + auto successAgeGauge = r.age_gauge("time.sinceLastSuccess"); + successAgeGauge.Set(1611081000); + + // Option 2: Create an Age Gauge from a MeterID + auto successMeter = r.new_id("time.sinceLastSuccess"); + r.age_gauge_with_id(successMeter).Set(1611081000); +} ``` -To set `now()` as the last success: +To set `Now()` as the last success: ```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create an Age Gauge + auto successAgeGauge = r.age_gauge("time.sinceLastSuccess"); + successAgeGauge.Set(1611081000); + // Option 2: Create an Age Gauge from a MeterID + auto successMeter = r.new_id("time.sinceLastSuccess"); + r.age_gauge_with_id(successMeter).Set(1611081000); +} ``` By default, a maximum of `1000` Age Gauges are allowed per `spectatord` process, because there is no diff --git a/docs/spectator/lang/cpp/meters/counter.md b/docs/spectator/lang/cpp/meters/counter.md index bb6942b83..25ddacb74 100644 --- a/docs/spectator/lang/cpp/meters/counter.md +++ b/docs/spectator/lang/cpp/meters/counter.md @@ -4,15 +4,43 @@ a Counter could be used to measure the rate at which it is being accessed. Counters are reported to the backend as a rate-per-second. In Atlas, the `:per-step` operator can be used to convert them back into a value-per-step on a graph. -Call `increment()` when an event occurs: +Call `Increment()` when an event occurs: ```cpp +#include +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Counter + auto serverRequestCounter = r.counter("server.numRequests"); + serverRequestCounter.Increment(); + + // Option 2: Create a Counter from a MeterID + auto serverRequestMeter = r.new_id("server.numRequests"); + r.counter_with_id(serverRequestMeter).Increment(); +} ``` -You can also pass a value to `increment()`. This is useful when a collection of events happens +You can also pass a value to `Increment()`. This is useful when a collection of events happens together: ```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Counter + auto serverRequestCounter = r.counter("server.numRequests"); + serverRequestCounter.Increment(10); + // Option 2: Create a Counter from a MeterID + auto serverRequestMeter = r.new_id("server.numRequests"); + r.counter_with_id(serverRequestMeter).Increment(10); +} ``` diff --git a/docs/spectator/lang/cpp/meters/dist-summary.md b/docs/spectator/lang/cpp/meters/dist-summary.md index c44ec8be0..af30ab548 100644 --- a/docs/spectator/lang/cpp/meters/dist-summary.md +++ b/docs/spectator/lang/cpp/meters/dist-summary.md @@ -6,8 +6,22 @@ Always use base units when recording data, to ensure that the tick labels presen are readable. If you are measuring payload size, then use bytes, not kilobytes (or some other unit). This means that a `4K` tick label will represent 4 kilobytes, rather than 4 kilo-kilobytes. -Call `record()` with a value: +Call `Record()` with a value: ```cpp +#include +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Distribution Summary + auto serverRequestSize = r.distribution_summary("server.requestSize"); + serverRequestSize.Record(42); + + // Option 2: Create a Distribution Summary from a MeterID + auto serverRequestMeter = r.new_id("server.requestSize"); + r.distribution_summary_with_id(serverRequestMeter).Record(42); +} ``` diff --git a/docs/spectator/lang/cpp/meters/gauge.md b/docs/spectator/lang/cpp/meters/gauge.md index 777de1157..f6265dc3a 100644 --- a/docs/spectator/lang/cpp/meters/gauge.md +++ b/docs/spectator/lang/cpp/meters/gauge.md @@ -6,10 +6,24 @@ Consider monitoring the behavior of a queue of tasks. If the data is being colle then a gauge for the size will show the size when it was sampled. The size may have been much higher or lower at some point during interval, but that is not known. -Call `set()` with a value: +Call `Set()` with a value: ```cpp +#include +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Gauge + auto serverQueueSize = r.gauge("server.queueSize"); + serverQueueSize.Set(10); + + // Option 2: Create a Gauge from a MeterID + auto serverQueueMeter = r.new_id("server.queueSize"); + r.gauge_with_id(serverQueueMeter).Set(10); +} ``` Gauges will report the last set value for 15 minutes. This done so that updates to the values do @@ -17,5 +31,20 @@ not need to be collected on a tight 1-minute schedule to ensure that Atlas shows graphs. A custom TTL may be configured for gauges. SpectatorD enforces a minimum TTL of 5 seconds. ```cpp +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Gauge + auto serverQueueSize = r.gauge("server.queueSize", {}, 120); + serverQueueSize.Set(10); + + // Option 2: Create a Gauge from a MeterID + auto serverQueueMeter = r.new_id("server.queueSize"); + r.gauge_with_id(serverQueueMeter, 120).Set(10); +} ``` diff --git a/docs/spectator/lang/cpp/meters/max-gauge.md b/docs/spectator/lang/cpp/meters/max-gauge.md index 6760ff44e..c3a020899 100644 --- a/docs/spectator/lang/cpp/meters/max-gauge.md +++ b/docs/spectator/lang/cpp/meters/max-gauge.md @@ -3,8 +3,22 @@ value to the backend. This ensures that only the maximum value observed during a is sent to the backend, thus over-riding the last-write-wins semantics of standard Gauges. Unlike standard Gauges, Max Gauges do not continue to report to the backend, and there is no TTL. -Call `set()` with a value: +Call `Set()` with a value: ```cpp +#include +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Max Gauge + auto serverQueueSize = r.max_gauge("server.queueSize"); + serverQueueSize.Set(10); + + // Option 2: Create a Gauge from a MeterID + auto serverQueueMeter = r.new_id("server.queueSize"); + r.max_gauge_with_id(serverQueueMeter).Set(10); +} ``` diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md index 5f13622f8..68cf28b41 100644 --- a/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md +++ b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md @@ -4,8 +4,22 @@ calculate a delta value and report it to the backend as a rate-per-second. A var metrics may be reported monotonically, and this metric type provides a convenient means of recording these values, at the expense of a slower time-to-first metric. -Call `set()` when an event occurs: +Call `Set()` when an event occurs: ```cpp +#include +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Monotonic Counter uint64_t + auto interfaceBytes = r.monotonic_counter_uint("iface.bytes"); + interfaceBytes.Set(10); + + // Option 2: Create a Monotonic Counter uint64_t from a MeterID + auto interfaceBytesMeter = r.new_id("iface.bytes"); + r.monotonic_counter_uint_with_id(interfaceBytesMeter).Set(10); +} ``` diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter.md b/docs/spectator/lang/cpp/meters/monotonic-counter.md index 991dbdab3..f8e95a695 100644 --- a/docs/spectator/lang/cpp/meters/monotonic-counter.md +++ b/docs/spectator/lang/cpp/meters/monotonic-counter.md @@ -4,8 +4,22 @@ calculate a delta value and report it to the backend as a rate-per-second. A var metrics may be reported monotonically, and this metric type provides a convenient means of recording these values, at the expense of a slower time-to-first metric. -Call `set()` when an event occurs: +Call `Set()` when an event occurs: ```cpp +#include +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Monotonic Counter + auto interfaceBytes = r.monotonic_counter("iface.bytes"); + interfaceBytes.Set(10); + + // Option 2: Create a Monotonic Counter from a MeterID + auto interfaceBytesMeter = r.new_id("iface.bytes"); + r.monotonic_counter_with_id(interfaceBytesMeter).Set(10); +} ``` diff --git a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md index bb65a24b4..7a8f1859e 100644 --- a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md +++ b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md @@ -8,8 +8,22 @@ In order to maintain the data distribution, they have a higher storage cost, wit up to 300X that of a standard Distribution Summary. Be diligent about any additional dimensions added to Percentile Distribution Summaries and ensure that they have a small bounded cardinality. -Call `record()` with a value: +Call `Record()` with a value: ```cpp +#include +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Percentile Distribution Summary + auto serverSize = r.pct_distribution_summary("server.requestSize"); + serverSize.Record(10); + + // Option 2: Create a Percentile Distribution Summary from a MeterID + auto requestSizeMeter = r.new_id("server.requestSize"); + r.pct_distribution_summary_with_id(requestSizeMeter).Record(10); +} ``` diff --git a/docs/spectator/lang/cpp/meters/percentile-timer.md b/docs/spectator/lang/cpp/meters/percentile-timer.md index 594f755c1..3273f39c0 100644 --- a/docs/spectator/lang/cpp/meters/percentile-timer.md +++ b/docs/spectator/lang/cpp/meters/percentile-timer.md @@ -8,17 +8,22 @@ In order to maintain the data distribution, they have a higher storage cost, wit up to 300X that of a standard Timer. Be diligent about any additional dimensions added to Percentile Timers and ensure that they have a small bounded cardinality. -Call `record()` with a value: +Call `Record()` with a value: ```cpp - -``` - -A `StopWatch` class is available, which may be used as a [Context Manager] to automatically record -the number of seconds that have elapsed while executing a block of code: - -```cpp - -``` - -[Context Manager]: https://docs.python.org/3/reference/datamodel.html#context-managers +#include + +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); + + // Option 1: Directly create a Percentile Timer + auto serverLatency = r.pct_timer("server.requestLatency"); + serverLatency.Record(10); + + // Option 2: Create a Percentile Timer from a MeterID + auto requestLatencyMeter = r.new_id("server.requestLatency"); + r.pct_timer_with_id(requestLatencyMeter).Record(10); +} +``` \ No newline at end of file diff --git a/docs/spectator/lang/cpp/meters/timer.md b/docs/spectator/lang/cpp/meters/timer.md index 1fe9e8675..894dc8d1b 100644 --- a/docs/spectator/lang/cpp/meters/timer.md +++ b/docs/spectator/lang/cpp/meters/timer.md @@ -1,16 +1,21 @@ A Timer is used to measure how long (in seconds) some event is taking. -Call `record()` with a value: +Call `Record()` with a value: ```cpp +#include -``` - -A `StopWatch` class is available, which may be used as a [Context Manager] to automatically record -the number of seconds that have elapsed while executing a block of code: +int main() +{ + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto r = Registry(config); -```cpp + // Option 1: Directly create a Timer + auto serverLatency = r.timer("server.requestLatency"); + serverLatency.Record(10); + // Option 2: Create a Timer from a MeterID + auto requestLatencyMeter = r.new_id("server.requestLatency"); + r.timer_with_id(requestLatencyMeter).Record(10); +} ``` - -[Context Manager]: https://docs.python.org/3/reference/datamodel.html#context-managers From 3b67d5fbf08abeba4f982e254a0388322b7e648d Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Fri, 11 Jul 2025 11:57:27 -0500 Subject: [PATCH 05/16] update work --- docs/spectator/lang/cpp/meters/age-gauge.md | 12 +- docs/spectator/lang/cpp/meters/counter.md | 12 +- .../spectator/lang/cpp/meters/dist-summary.md | 6 +- docs/spectator/lang/cpp/meters/gauge.md | 12 +- docs/spectator/lang/cpp/meters/max-gauge.md | 6 +- .../lang/cpp/meters/monotonic-counter-uint.md | 6 +- .../lang/cpp/meters/monotonic-counter.md | 6 +- .../cpp/meters/percentile-dist-summary.md | 6 +- .../lang/cpp/meters/percentile-timer.md | 6 +- docs/spectator/lang/cpp/meters/timer.md | 6 +- docs/spectator/lang/cpp/migrations.md | 41 ++++++ docs/spectator/lang/cpp/usage.md | 138 ++++++++++++------ 12 files changed, 175 insertions(+), 82 deletions(-) diff --git a/docs/spectator/lang/cpp/meters/age-gauge.md b/docs/spectator/lang/cpp/meters/age-gauge.md index 9d5c850df..5e0270164 100644 --- a/docs/spectator/lang/cpp/meters/age-gauge.md +++ b/docs/spectator/lang/cpp/meters/age-gauge.md @@ -15,12 +15,12 @@ int main() auto r = Registry(config); // Option 1: Directly create an Age Gauge - auto successAgeGauge = r.age_gauge("time.sinceLastSuccess"); + auto successAgeGauge = r.CreateAgeGauge("time.sinceLastSuccess"); successAgeGauge.Set(1611081000); // Option 2: Create an Age Gauge from a MeterID - auto successMeter = r.new_id("time.sinceLastSuccess"); - r.age_gauge_with_id(successMeter).Set(1611081000); + auto successMeter = r.CreateNewId("time.sinceLastSuccess"); + r.CreateAgeGauge(successMeter).Set(1611081000); } ``` @@ -35,12 +35,12 @@ int main() auto r = Registry(config); // Option 1: Directly create an Age Gauge - auto successAgeGauge = r.age_gauge("time.sinceLastSuccess"); + auto successAgeGauge = r.CreateAgeGauge("time.sinceLastSuccess"); successAgeGauge.Set(1611081000); // Option 2: Create an Age Gauge from a MeterID - auto successMeter = r.new_id("time.sinceLastSuccess"); - r.age_gauge_with_id(successMeter).Set(1611081000); + auto successMeter = r.CreateNewId("time.sinceLastSuccess"); + r.CreateAgeGauge(successMeter).Set(1611081000); } ``` diff --git a/docs/spectator/lang/cpp/meters/counter.md b/docs/spectator/lang/cpp/meters/counter.md index 25ddacb74..75525cfe8 100644 --- a/docs/spectator/lang/cpp/meters/counter.md +++ b/docs/spectator/lang/cpp/meters/counter.md @@ -15,12 +15,12 @@ int main() auto r = Registry(config); // Option 1: Directly create a Counter - auto serverRequestCounter = r.counter("server.numRequests"); + auto serverRequestCounter = r.CreateCounter("server.numRequests"); serverRequestCounter.Increment(); // Option 2: Create a Counter from a MeterID - auto serverRequestMeter = r.new_id("server.numRequests"); - r.counter_with_id(serverRequestMeter).Increment(); + auto serverRequestMeter = r.CreateNewId("server.numRequests"); + r.CreateCounter(serverRequestMeter).Increment(); } ``` @@ -36,11 +36,11 @@ int main() auto r = Registry(config); // Option 1: Directly create a Counter - auto serverRequestCounter = r.counter("server.numRequests"); + auto serverRequestCounter = r.CreateCounter("server.numRequests"); serverRequestCounter.Increment(10); // Option 2: Create a Counter from a MeterID - auto serverRequestMeter = r.new_id("server.numRequests"); - r.counter_with_id(serverRequestMeter).Increment(10); + auto serverRequestMeter = r.CreateNewId("server.numRequests"); + r.CreateCounter(serverRequestMeter).Increment(10); } ``` diff --git a/docs/spectator/lang/cpp/meters/dist-summary.md b/docs/spectator/lang/cpp/meters/dist-summary.md index af30ab548..ccf75ed57 100644 --- a/docs/spectator/lang/cpp/meters/dist-summary.md +++ b/docs/spectator/lang/cpp/meters/dist-summary.md @@ -17,11 +17,11 @@ int main() auto r = Registry(config); // Option 1: Directly create a Distribution Summary - auto serverRequestSize = r.distribution_summary("server.requestSize"); + auto serverRequestSize = r.CreateDistributionSummary("server.requestSize"); serverRequestSize.Record(42); // Option 2: Create a Distribution Summary from a MeterID - auto serverRequestMeter = r.new_id("server.requestSize"); - r.distribution_summary_with_id(serverRequestMeter).Record(42); + auto serverRequestMeter = r.CreateNewId("server.requestSize"); + r.CreateDistributionSummary(serverRequestMeter).Record(42); } ``` diff --git a/docs/spectator/lang/cpp/meters/gauge.md b/docs/spectator/lang/cpp/meters/gauge.md index f6265dc3a..afb7b8076 100644 --- a/docs/spectator/lang/cpp/meters/gauge.md +++ b/docs/spectator/lang/cpp/meters/gauge.md @@ -17,12 +17,12 @@ int main() auto r = Registry(config); // Option 1: Directly create a Gauge - auto serverQueueSize = r.gauge("server.queueSize"); + auto serverQueueSize = r.CreateGauge("server.queueSize"); serverQueueSize.Set(10); // Option 2: Create a Gauge from a MeterID - auto serverQueueMeter = r.new_id("server.queueSize"); - r.gauge_with_id(serverQueueMeter).Set(10); + auto serverQueueMeter = r.CreateNewId("server.queueSize"); + r.CreateGauge(serverQueueMeter).Set(10); } ``` @@ -39,12 +39,12 @@ int main() auto r = Registry(config); // Option 1: Directly create a Gauge - auto serverQueueSize = r.gauge("server.queueSize", {}, 120); + auto serverQueueSize = r.CreateGauge("server.queueSize", {}, 120); serverQueueSize.Set(10); // Option 2: Create a Gauge from a MeterID - auto serverQueueMeter = r.new_id("server.queueSize"); - r.gauge_with_id(serverQueueMeter, 120).Set(10); + auto serverQueueMeter = r.CreateNewId("server.queueSize"); + r.CreateGauge(serverQueueMeter, 120).Set(10); } ``` diff --git a/docs/spectator/lang/cpp/meters/max-gauge.md b/docs/spectator/lang/cpp/meters/max-gauge.md index c3a020899..5712b7ecc 100644 --- a/docs/spectator/lang/cpp/meters/max-gauge.md +++ b/docs/spectator/lang/cpp/meters/max-gauge.md @@ -14,11 +14,11 @@ int main() auto r = Registry(config); // Option 1: Directly create a Max Gauge - auto serverQueueSize = r.max_gauge("server.queueSize"); + auto serverQueueSize = r.CreateMaxGauge("server.queueSize"); serverQueueSize.Set(10); // Option 2: Create a Gauge from a MeterID - auto serverQueueMeter = r.new_id("server.queueSize"); - r.max_gauge_with_id(serverQueueMeter).Set(10); + auto serverQueueMeter = r.CreateNewId("server.queueSize"); + r.CreateMaxGauge(serverQueueMeter).Set(10); } ``` diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md index 68cf28b41..0a772f5be 100644 --- a/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md +++ b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md @@ -15,11 +15,11 @@ int main() auto r = Registry(config); // Option 1: Directly create a Monotonic Counter uint64_t - auto interfaceBytes = r.monotonic_counter_uint("iface.bytes"); + auto interfaceBytes = r.CreateMonotonicCounterUint("iface.bytes"); interfaceBytes.Set(10); // Option 2: Create a Monotonic Counter uint64_t from a MeterID - auto interfaceBytesMeter = r.new_id("iface.bytes"); - r.monotonic_counter_uint_with_id(interfaceBytesMeter).Set(10); + auto interfaceBytesMeter = r.CreateNewId("iface.bytes"); + r.CreateMonotonicCounterUint(interfaceBytesMeter).Set(10); } ``` diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter.md b/docs/spectator/lang/cpp/meters/monotonic-counter.md index f8e95a695..b127102d1 100644 --- a/docs/spectator/lang/cpp/meters/monotonic-counter.md +++ b/docs/spectator/lang/cpp/meters/monotonic-counter.md @@ -15,11 +15,11 @@ int main() auto r = Registry(config); // Option 1: Directly create a Monotonic Counter - auto interfaceBytes = r.monotonic_counter("iface.bytes"); + auto interfaceBytes = r.CreateMonotonicCounter("iface.bytes"); interfaceBytes.Set(10); // Option 2: Create a Monotonic Counter from a MeterID - auto interfaceBytesMeter = r.new_id("iface.bytes"); - r.monotonic_counter_with_id(interfaceBytesMeter).Set(10); + auto interfaceBytesMeter = r.CreateNewId("iface.bytes"); + r.CreateMonotonicCounter(interfaceBytesMeter).Set(10); } ``` diff --git a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md index 7a8f1859e..2adbb99e0 100644 --- a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md +++ b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md @@ -19,11 +19,11 @@ int main() auto r = Registry(config); // Option 1: Directly create a Percentile Distribution Summary - auto serverSize = r.pct_distribution_summary("server.requestSize"); + auto serverSize = r.CreatePercentDistributionSummary("server.requestSize"); serverSize.Record(10); // Option 2: Create a Percentile Distribution Summary from a MeterID - auto requestSizeMeter = r.new_id("server.requestSize"); - r.pct_distribution_summary_with_id(requestSizeMeter).Record(10); + auto requestSizeMeter = r.CreateNewId("server.requestSize"); + r.CreatePercentDistributionSummary(requestSizeMeter).Record(10); } ``` diff --git a/docs/spectator/lang/cpp/meters/percentile-timer.md b/docs/spectator/lang/cpp/meters/percentile-timer.md index 3273f39c0..b0c3b7aa0 100644 --- a/docs/spectator/lang/cpp/meters/percentile-timer.md +++ b/docs/spectator/lang/cpp/meters/percentile-timer.md @@ -19,11 +19,11 @@ int main() auto r = Registry(config); // Option 1: Directly create a Percentile Timer - auto serverLatency = r.pct_timer("server.requestLatency"); + auto serverLatency = r.CreatePercentTimer("server.requestLatency"); serverLatency.Record(10); // Option 2: Create a Percentile Timer from a MeterID - auto requestLatencyMeter = r.new_id("server.requestLatency"); - r.pct_timer_with_id(requestLatencyMeter).Record(10); + auto requestLatencyMeter = r.CreateNewId("server.requestLatency"); + r.CreatePercentTimer(requestLatencyMeter).Record(10); } ``` \ No newline at end of file diff --git a/docs/spectator/lang/cpp/meters/timer.md b/docs/spectator/lang/cpp/meters/timer.md index 894dc8d1b..d3d4e11c7 100644 --- a/docs/spectator/lang/cpp/meters/timer.md +++ b/docs/spectator/lang/cpp/meters/timer.md @@ -11,11 +11,11 @@ int main() auto r = Registry(config); // Option 1: Directly create a Timer - auto serverLatency = r.timer("server.requestLatency"); + auto serverLatency = r.CreateTimer("server.requestLatency"); serverLatency.Record(10); // Option 2: Create a Timer from a MeterID - auto requestLatencyMeter = r.new_id("server.requestLatency"); - r.timer_with_id(requestLatencyMeter).Record(10); + auto requestLatencyMeter = r.CreateNewId("server.requestLatency"); + r.CreateTimer(requestLatencyMeter).Record(10); } ``` diff --git a/docs/spectator/lang/cpp/migrations.md b/docs/spectator/lang/cpp/migrations.md index e69de29bb..0f4cca21e 100644 --- a/docs/spectator/lang/cpp/migrations.md +++ b/docs/spectator/lang/cpp/migrations.md @@ -0,0 +1,41 @@ +## Migrating to 2.X + +Version 2.X consists of a major rewrite that greatly simplifies spectator-cpp and the process in which it sends metrics to SpectatorD + +### New + +#### Writers + +`spectator.Registry` now supports 3 different writers. The WriterType is specefied through a WriterConfig object. + +See [Usage > Output Location](usage.md#output-location) for more details. + +#### Common Tags + +A few local environment common tags are now automatically added to all Meters. Their values are read +from the environment variables. + +| Tag | Environment Variable | +|--------------|----------------------| +| nf.container | TITUS_CONTAINER_NAME | +| nf.process | NETFLIX_PROCESS_NAME | + +Tags from environment variables take precedence over tags passed on code when creating the `Config`. + +Note that common tags sourced by [spectatord](https://github.com/Netflix-Skunkworks/spectatord) can't be overwritten. + +#### Registry, Config, and Writer Config + +* `Config` is now created through a constructor which throws error if the passed in parameters are not valid. +* `Config` members are now private. + +### Moved + + + +### Removed + + + +### Migration Steps + diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index db8e6ac18..f5297cca8 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -12,7 +12,7 @@ This library currently utilzes C++ 20. ## Installing & Building -If your project utilizes CMake you can incorporate this project by simply calling add_subdirectory(spectator-cpp). +If your project utilizes CMake you can incorporate this project by simply calling CMake's add_subdirectory() command on the root folder of the project. If you wish to simply build the Spectator-CPP thin client you can utilize the Docker containter instructions found here https://github.com/Netflix/spectator-cpp/tree/main/Dockerfiles. The container installs a minimal set of depencies to build the project in the container, such as g++-13, python3, and conan. The project only has 3 external dependencies @@ -37,14 +37,14 @@ int main() auto r = Registry(config); // Create some meters - auto threadGauge = r.gauge("threads"); - auto queueGauge = r.gauge("queue-size", {{"my-tags", "bar"}}); + auto threadGauge = r.CreateGauge("threads"); + auto queueGauge = r.CreateGauge("queue-size", {{"my-tags", "bar"}}); auto memoryWriter = static_cast(WriterTestHelper::GetImpl()); EXPECT_TRUE(memoryWriter->IsEmpty()); threadGauge.Set(GetNumThreads()); // Metric sent to SpectatorD: "g:threads,platform=my-platform,process=my-process:5.000000\n" - queueGauge.Set(GetQueueSize()); // Metric sent to SpectatorD: "g:queue-size,my-tags=bar,platform=my-platform,process=my-process:10.000000\n" + queueGauge.Set(GetQueueSize()); // Metric sent to SpectatorD: "g:queue-size,my-tags=bar,platform=my-platform,process=my-process:10.000000\n" } ``` @@ -64,12 +64,12 @@ Coming Soon ## Working with MeterId Objects Each metric stored in Atlas is uniquely identified by the combination of the name and the tags -associated with it. In `spectator-cpp`, this data is represented with `Id` objects, created -by the `Registry`. The `NewId()` method returns new `Id` objects, which have extra common +associated with it. In `spectator-cpp`, this data is represented with `MeterId` objects, created +by the `Registry`. The `CreateNewId()` method returns new a `MeterId` object, which have extra common tags applied, and which can be further customized by calling the `WithTag()` and `WithTags()` -methods. Each `Id` will create and store a validated subset of the `spectatord` protocol line -to be written for each `Meter`, when it is instantiated. `Id` objects can be passed around and -used concurrently. Manipulating the tags with the provided methods will create new `Id` objects. +methods. Each `MeterId` will create and store a validated subset of the `spectatord` protocol line +to be written for each `Meter`, when it is instantiated. Manipulating the tags with the provided methods + will create new `MeterId` objects. Note that **all tag keys and values must be strings.** For example, if you want to keep track of the number of successful requests, then you must cast integers to strings. The `Id` class will @@ -82,21 +82,22 @@ validate these values, dropping or changing any that are not valid, and reportin int main() { + // Create common tags std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; + // Initialize the Registry auto config = Config(WriterConfig(WriterTypes::Memory), commonTags); auto registry = Registry(config); - registry.counter("server.requests", {{"statusCode", std::to_string(200)}}).Increment(); - // Option 1: Using the registry to create a MeterId - auto numRequestsId = registry.new_id("server.numRequests", {{"statusCode", std::to_string(200)}}); - // Creating the counter with the MeterId and Incrementing it - registry.counter_with_id(numRequestsId).Increment(); + // Option 1: Using the registry to create a MeterId & creating a Counter from the MeterId + auto numRequestsId = registry.CreateNewId("server.numRequests", {{"statusCode", std::to_string(200)}}); + registry.CreateCounter(numRequestsId).Increment(); // Option 2: Directly creating a Counter - auto numRequestsCounter = registry.counter("server.numRequests", {{"statusCode", std::to_string(200)}}); + auto numRequestsCounter = registry.CreateCounter("server.numRequests2", {{"statusCode", std::to_string(200)}}); numRequestsCounter.Increment(); + } ``` @@ -124,39 +125,70 @@ for general guidelines on metrics naming and restrictions. ## Output Location `spectator.Registry` now supports three different writers. The writer types are Memory Writer, UDP Writer, -and a UDS (Unix Domain Socket) Writer +and a UDS (Unix Domain Socket) Writer. In order to define the writer type you must initialize the Registry +with a Config object. A Config Object takes a required WriterConfig object and optional extra tags. A WriterConfig object +takes a location and an optional buffering parameter. Buffering is allowed for all writer types. + +```cpp +// Writer Config Constructors + +// Option 1: Define a location and no buffering +WriterConfig(const std::string& type) + +// Option 2: Define a location with buffering +WriterConfig(const std::string& type, unsigned int bufferSize); -Possible values are: -* `""` - Empty string will default to `udp`, with the `LineBuffer` disabled by default. -* `none` - A no-op writer that does nothing. Used to disable metrics collection. -* `memory` - Write to memory. Useful for testing. -* `stderr` - Write to standard error for the process. -* `stdout` - Write to standard output for the process. -* `udp` - Write to the default UDP port for `spectatord`. This is the default location. -* `unix` - Write to the default Unix Domain Socket for `spectatord`. Useful for high-volume scenarios. -* `file:///path/to/file` - Write to a custom file (e.g. `file:///tmp/foo/bar`). -* `udp://host:port` - Write to a custom UDP socket (e.g. `udp://127.0.0.1:1235`). -* `unix:///path/to/socket` - Write to a custom Unix domain socket (e.g. `unix:///tmp/some.socket`). +// Config Constructor +// Required WriterConfig with optional extraTags to be applied to all metrics +Config(const WriterConfig& writerConfig, const std::unordered_map& extraTags = {}); +``` + +{% raw %} + +```cpp +// Default Writer Config Examples +WriterConfig wConfig(WriterTypes::Memory); // write metrics to memory for testing +WriterConfig wConfig(WriterTypes::UDP); // the default UDP address for `spectatord`. +WriterConfig wConfig(WriterTypes::Unix, 4096); // the default unix address for spectatord with buffering + +// Custom Writer Config Location Examples +const std::string udpUrl = std::string(WriterTypes::UDPURL) + "192.168.1.100:8125"; +const WriterConfig wConfig(udpUrl); + +const std::string unixUrl = std::string(WriterTypes::UnixURL) + "/var/run/custom/socket.sock"; +const WriterConfig wConfig(unixUrl, 4096); + +// Config Examples +Config config = Config(WriterConfig(WriterTypes::Memory)); + +std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; +Config config = Config(WriterConfig(WriterTypes::Memory), commonTags); + +// Registry Initialization +WriterConfig wConfig(WriterTypes::Memory); // write metrics to memory for testing +Config config = Config(wConfig); +Registry registry(config); +``` +{% endraw %} Location can also be set through the environment variable `SPECTATOR_OUTPUT_LOCATION`. If both are set, -the environment variable takes precedence over the passed config. If either values provided to the WriterConfig are +the environment variable takes precedence over the value passed to the WriterConfig. If either values provided to the WriterConfig are invalid a runtime exception will be thrown. ## Line Buffer -The `NewConfigWithBuffer` factory function takes a `bufferSize` parameter that configures an optional -`LineBuffer`, which caches protocol lines locally, before flushing them to `spectatord`. Flushes occur -under two conditions: (1) the buffer size is exceeded, or (2) five seconds has elapsed. The buffer is -available for the `UdpWriter` and the `UnixgramWriter`, where performance matters most. The `LineBuffer` -is disabled by default (with size zero) in the standard `NewConfig` factory function, to ensure that the -default operation of the library works under most circumstances. For high-performance scenarios, a 60KB +The WriterConfig has an optional to set a default `bufferSize` parameter. If this parameter is not set each time +a meter is recorded, that meter will send the metric to spectatord using the writer type defined in your WriterConfig. +If the meter is set the buffer will cache protocol lines locally before flushing them to `spectatord`. Flushes occur only +under one condition that the buffer size has been exceeded.or high-performance scenarios, a 60KB buffer size is recommended. The maximum buffer size for udp sockets and unix domain sockets on Linux is 64KB, so stay under this limit. + ## Batch Usage -When using `spectator-go` to report metrics from a batch job, ensure that the batch job runs for at +When using `spectator-cpp` to report metrics from a batch job, ensure that the batch job runs for at least five (5), if not ten (10) seconds in duration. This is necessary in order to allow sufficient time for `spectatord` to publish metrics to the Atlas backend; it publishes every five seconds. If your job does not run this long, or you find you are missing metrics that were reported at the end @@ -188,23 +220,43 @@ scenarios. To write tests against this library, instantiate a test instance of the `Registry` and configure it to use the [MemoryWriter](https://github.com/Netflix/spectator-go/blob/main/spectator/writer/writer.go#L18-L21), -which stores all updates in an `Array`. Maintain a handle to the `MemoryWriter`, then inspect the +which stores all updates in an `Vector`. Maintain a handle to the `MemoryWriter`, then inspect the `Lines()` to verify your metrics updates. See the source code for more testing examples. -### Protocol Parser +{% raw %} + +```cpp +int main() +{ + // Initialize Registry + auto config = Config(WriterConfig(WriterTypes::Memory)); + auto registry = Registry(config); -A [SpectatorD] line protocol parser is available, which can be used for validating the results -captured by a `MemoryWriter`. + // Directly create a Counter + auto numRequestsCounter = registry.CreateCounter("server.numRequests2", {{"statusCode", std::to_string(200)}}); -[SpectatorD]: ../../agent/usage.md + // Create a handle to the Writer + auto memoryWriter = static_cast(WriterTestHelper::GetImpl()); + + + numRequestsCounter.Increment(); + + auto messages = memoryWriter->GetMessages(); + for (const auto& message : messages) + { + std::cout << message; // Print all messages sent to SpectatorD + } +} +``` +{% endraw %} ## Performance -On an `m5d.2xlarge` EC2 instance, with `Go 1.24.3` and `github.com/Netflix/spectator-go/v2 v2.0.13`, we +On an `m5d.2xlarge` EC2 instance, with `spectator-cpp-2.0` and `github.com/Netflix/spectator-cpp/v2 v2.0.13`, we have observed the following single-threaded performance numbers across a two-minute test window: -* 135,771 requests/second over `udp` -* 206,641 requests/second over `unix` +* requests/second over `udp` +* requests/second over `unix` The benchmark incremented a single counter with two tags in a tight loop, to simulate real-world tag usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol line From 7ea72fe4e72537d4538f2fde6d476f46a227b8ff Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Fri, 11 Jul 2025 17:11:49 -0500 Subject: [PATCH 06/16] update work --- docs/spectator/lang/cpp/meters/age-gauge.md | 4 +- .../lang/cpp/meters/percentile-timer.md | 2 +- docs/spectator/lang/cpp/usage.md | 154 ++++++++++-------- 3 files changed, 93 insertions(+), 67 deletions(-) diff --git a/docs/spectator/lang/cpp/meters/age-gauge.md b/docs/spectator/lang/cpp/meters/age-gauge.md index 5e0270164..f13ba34d6 100644 --- a/docs/spectator/lang/cpp/meters/age-gauge.md +++ b/docs/spectator/lang/cpp/meters/age-gauge.md @@ -36,11 +36,11 @@ int main() // Option 1: Directly create an Age Gauge auto successAgeGauge = r.CreateAgeGauge("time.sinceLastSuccess"); - successAgeGauge.Set(1611081000); + successAgeGauge.Now(); // Option 2: Create an Age Gauge from a MeterID auto successMeter = r.CreateNewId("time.sinceLastSuccess"); - r.CreateAgeGauge(successMeter).Set(1611081000); + r.CreateAgeGauge(successMeter).Now(); } ``` diff --git a/docs/spectator/lang/cpp/meters/percentile-timer.md b/docs/spectator/lang/cpp/meters/percentile-timer.md index b0c3b7aa0..b28f3b878 100644 --- a/docs/spectator/lang/cpp/meters/percentile-timer.md +++ b/docs/spectator/lang/cpp/meters/percentile-timer.md @@ -26,4 +26,4 @@ int main() auto requestLatencyMeter = r.CreateNewId("server.requestLatency"); r.CreatePercentTimer(requestLatencyMeter).Record(10); } -``` \ No newline at end of file +``` diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index f5297cca8..fd24cb5ed 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -12,11 +12,12 @@ This library currently utilzes C++ 20. ## Installing & Building -If your project utilizes CMake you can incorporate this project by simply calling CMake's add_subdirectory() command on the root folder of the project. -If you wish to simply build the Spectator-CPP thin client you can utilize the Docker containter instructions found -here https://github.com/Netflix/spectator-cpp/tree/main/Dockerfiles. The container installs a minimal set of depencies -to build the project in the container, such as g++-13, python3, and conan. The project only has 3 external dependencies -spdlog, gtest and boost. These dependencies are managed through conan. +If your project uses CMake, you can easily integrate this library by calling `add_subdirectory()` +on the root folder. To build the Spectator-CPP thin client independently, follow the Docker +container instructions at [Dockerfiles](https://github.com/Netflix/spectator-cpp/tree/main/Dockerfiles). +The container provides a minimal build environment with g++-13, python3, and conan. Spectator-CPP +relies on just three external dependencies—spdlog, gtest, and boost—which are managed automatically +via conan. ## Instrumenting Code @@ -40,11 +41,13 @@ int main() auto threadGauge = r.CreateGauge("threads"); auto queueGauge = r.CreateGauge("queue-size", {{"my-tags", "bar"}}); - auto memoryWriter = static_cast(WriterTestHelper::GetImpl()); - EXPECT_TRUE(memoryWriter->IsEmpty()); + threadGauge.Set(GetNumThreads()); + queueGauge.Set(GetQueueSize()); - threadGauge.Set(GetNumThreads()); // Metric sent to SpectatorD: "g:threads,platform=my-platform,process=my-process:5.000000\n" - queueGauge.Set(GetQueueSize()); // Metric sent to SpectatorD: "g:queue-size,my-tags=bar,platform=my-platform,process=my-process:10.000000\n" + /* Metrics Sent: + "g:threads,platform=my-platform,process=my-process:5.000000\n" + "g:queue-size,my-tags=bar,platform=my-platform,process=my-process:10.000000\n" + */ } ``` @@ -52,15 +55,15 @@ int main() ## Logging -Logging is implemented with spdlog and the default location is standard output. The default log level is spdlog::level::info. The Logger class is a singleton -and provides a function Logger::GetLogger(). You can change the logger level by simply calling Logger::GetLogger()->set_level(spdlog::level::debug); as long as -the logger has been created successfully +Logging uses the `spdlog` library and outputs to standard output by default, with a default log +level of `spdlog::level::info`. The `Logger` class is a singleton and provides the +`Logger::GetLogger()` function to access the logger instance. To change the log level, call +`Logger::GetLogger()->set_level(spdlog::level::debug);` after the logger has been successfully created. ## Runtime Metrics Coming Soon - ## Working with MeterId Objects Each metric stored in Atlas is uniquely identified by the combination of the name and the tags @@ -97,7 +100,6 @@ int main() // Option 2: Directly creating a Counter auto numRequestsCounter = registry.CreateCounter("server.numRequests2", {{"statusCode", std::to_string(200)}}); numRequestsCounter.Increment(); - } ``` @@ -122,69 +124,94 @@ for general guidelines on metrics naming and restrictions. * [Percentile Timer](./meters/percentile-timer.md) * [Timer](./meters/timer.md) -## Output Location +## Output Locations -`spectator.Registry` now supports three different writers. The writer types are Memory Writer, UDP Writer, -and a UDS (Unix Domain Socket) Writer. In order to define the writer type you must initialize the Registry -with a Config object. A Config Object takes a required WriterConfig object and optional extra tags. A WriterConfig object -takes a location and an optional buffering parameter. Buffering is allowed for all writer types. +`spectator.Registry` supports three output writer types: Memory Writer, UDP Writer, and Unix Domain +Socket (UDS) Writer. To specify the writer type, initialize the Registry with a `Config` object. A +`Config` requires a `WriterConfig` (which defines the writer type and location) and can optionally +include extra tags to be applied to all metrics. The `WriterConfig` also accepts an optional +buffer size parameter, enabling buffering for all writer types. -```cpp -// Writer Config Constructors +### Writer Config Constructors -// Option 1: Define a location and no buffering +```cpp +// Constructor 1: Define a location and no buffering WriterConfig(const std::string& type) -// Option 2: Define a location with buffering +// Constructor 2: Define a location with buffering WriterConfig(const std::string& type, unsigned int bufferSize); +``` +### Writer Config Examples -// Config Constructor -// Required WriterConfig with optional extraTags to be applied to all metrics -Config(const WriterConfig& writerConfig, const std::unordered_map& extraTags = {}); +{% raw %} + +```cpp +/* Default Writer Config Examples */ + +// Write metrics to memory for testing +WriterConfig wConfig(WriterTypes::Memory); + +// Default UDP address for spectatord +WriterConfig wConfig(WriterTypes::UDP); + +// Default UDS address for spectatord with buffering +WriterConfig wConfig(WriterTypes::Unix, 4096); + +/* Custom Writer Config Location Examples */ + +// Custom UDP writer location +std::string udpUrl = std::string(WriterTypes::UDPURL) + "192.168.1.100:8125"; +WriterConfig wConfig(udpUrl); + +// Custom UDS writer location +std::string unixUrl = std::string(WriterTypes::UnixURL) + "/var/run/custom/socket.sock"; +WriterConfig wConfig(unixUrl, 4096); ``` -{% raw %} +{% endraw %} + +### Config Constructor ```cpp -// Default Writer Config Examples -WriterConfig wConfig(WriterTypes::Memory); // write metrics to memory for testing -WriterConfig wConfig(WriterTypes::UDP); // the default UDP address for `spectatord`. -WriterConfig wConfig(WriterTypes::Unix, 4096); // the default unix address for spectatord with buffering +// Constructor: WriterConfig & optional extraTags for all metrics +Config(const WriterConfig& writerConfig, const std::unordered_map& extraTags = {}); +``` -// Custom Writer Config Location Examples -const std::string udpUrl = std::string(WriterTypes::UDPURL) + "192.168.1.100:8125"; -const WriterConfig wConfig(udpUrl); +### Config Examples -const std::string unixUrl = std::string(WriterTypes::UnixURL) + "/var/run/custom/socket.sock"; -const WriterConfig wConfig(unixUrl, 4096); +{% raw %} + +```cpp +/* Config Examples */ -// Config Examples +// Config with a WriterConfig & no extra tags Config config = Config(WriterConfig(WriterTypes::Memory)); +// Config with a WriterConfig & extra tags std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; Config config = Config(WriterConfig(WriterTypes::Memory), commonTags); -// Registry Initialization -WriterConfig wConfig(WriterTypes::Memory); // write metrics to memory for testing +/* Registry Initialization */ +WriterConfig wConfig(WriterTypes::Memory); Config config = Config(wConfig); Registry registry(config); ``` + {% endraw %} -Location can also be set through the environment variable `SPECTATOR_OUTPUT_LOCATION`. If both are set, -the environment variable takes precedence over the value passed to the WriterConfig. If either values provided to the WriterConfig are -invalid a runtime exception will be thrown. +Location can also be set through the environment variable `SPECTATOR_OUTPUT_LOCATION`. If both are +set, the environment variable takes precedence over the value passed to the WriterConfig. If either +values provided to the WriterConfig are invalid a runtime exception will be thrown. ## Line Buffer -The WriterConfig has an optional to set a default `bufferSize` parameter. If this parameter is not set each time -a meter is recorded, that meter will send the metric to spectatord using the writer type defined in your WriterConfig. -If the meter is set the buffer will cache protocol lines locally before flushing them to `spectatord`. Flushes occur only -under one condition that the buffer size has been exceeded.or high-performance scenarios, a 60KB -buffer size is recommended. The maximum buffer size for udp sockets and unix domain sockets on Linux is -64KB, so stay under this limit. - +The `WriterConfig` allows you to set an optional `bufferSize` parameter. If `bufferSize` is not +set, each metric is sent immediately to `spectatord` using the configured writer type. If +`bufferSize` is set, metrics are buffered locally and only flushed to `spectatord` when the buffer +exceeds the specified size. For high-performance scenarios, a buffer size of 60KB is recommended. +The maximum buffer size for UDP and Unix Domain Socket writers on Linux is 64KB, so ensure your +buffer size does not exceed this limit. ## Batch Usage @@ -219,9 +246,9 @@ scenarios. ## Writing Tests To write tests against this library, instantiate a test instance of the `Registry` and configure it -to use the [MemoryWriter](https://github.com/Netflix/spectator-go/blob/main/spectator/writer/writer.go#L18-L21), -which stores all updates in an `Vector`. Maintain a handle to the `MemoryWriter`, then inspect the -`Lines()` to verify your metrics updates. See the source code for more testing examples. +to use the `MemoryWriter`, which stores all updates in a `Vector`. Maintain a handle to the +`MemoryWriter`, then inspect the protocol lines with `GetMessages()` to verify your metric updates. +See the source code for more testing examples. {% raw %} @@ -238,29 +265,28 @@ int main() // Create a handle to the Writer auto memoryWriter = static_cast(WriterTestHelper::GetImpl()); - numRequestsCounter.Increment(); auto messages = memoryWriter->GetMessages(); - for (const auto& message : messages) - { + for (const auto& message : messages) { std::cout << message; // Print all messages sent to SpectatorD } } ``` + {% endraw %} ## Performance -On an `m5d.2xlarge` EC2 instance, with `spectator-cpp-2.0` and `github.com/Netflix/spectator-cpp/v2 v2.0.13`, we -have observed the following single-threaded performance numbers across a two-minute test window: +On an `m5d.2xlarge` EC2 instance, with `spectator-cpp-2.0` and +`github.com/Netflix/spectator-cpp/v2 v2.0.13`, we have observed the following single-threaded +performance numbers across a two-minute test window: -* requests/second over `udp` -* requests/second over `unix` +* requests/second over `udp` +* requests/second over `unix` -The benchmark incremented a single counter with two tags in a tight loop, to simulate real-world tag -usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol line -was `74` characters in length. +The benchmark incremented a single counter with two tags in a tight loop, to simulate real-world +tag usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol +line was `74` characters in length. -The Go process CPU usage was ~112% and the `spectatord` process CPU usage was ~62% on this 8 vCPU -system, for `udp`. It was ~113% and ~85%, respectively, for `unix`. +The Go process CPU usage was ~112% and the `spectatord` process CPU usage was ~62% on this 8 vCPU system, for `udp`. It was ~113% and ~85%, respectively, for `unix`. From 84f7f445a27dc8f1db2b701d20499752da54d3a8 Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Sun, 13 Jul 2025 17:56:57 -0500 Subject: [PATCH 07/16] update performance test --- docs/spectator/lang/cpp/perf-test.md | 22 +++++++++++----------- docs/spectator/lang/cpp/usage.md | 8 +++----- 2 files changed, 14 insertions(+), 16 deletions(-) diff --git a/docs/spectator/lang/cpp/perf-test.md b/docs/spectator/lang/cpp/perf-test.md index 6e258d259..7ff213e00 100644 --- a/docs/spectator/lang/cpp/perf-test.md +++ b/docs/spectator/lang/cpp/perf-test.md @@ -16,30 +16,30 @@ Test maximum single-threaded throughput for two minutes. int main() { Logger::info("Starting UDP performance test..."); - - auto r = Registry(Config(WriterConfig(WriterTypes::UDP))); - //auto r = Registry(Config(WriterConfig(WriterTypes::Unix))); - + + //auto r = Registry(Config(WriterConfig(WriterTypes::UDP))); + auto r = Registry(Config(WriterConfig(WriterTypes::Unix))); + std::unordered_map tags = { {"location", "udp"}, {"version", "correct-horse-battery-staple"}}; // Set maximum duration to 2 minutes constexpr int max_duration_seconds = 2 * 60; - + // Track iterations and timing unsigned long long iterations = 0; auto start_time = std::chrono::steady_clock::now(); - + // Helper function to get elapsed time in seconds auto elapsed = [&start_time]() -> double { auto now = std::chrono::steady_clock::now(); return std::chrono::duration(now - start_time).count(); }; - + while (true) { - r.counter("udp_test_counter", tags).Increment(); + r.CreateCounter("udp_test_counter", tags).Increment(); iterations++; - + if (iterations % 500000 == 0) { if (elapsed() > max_duration_seconds) @@ -48,10 +48,10 @@ int main() } } } - + double total_elapsed = elapsed(); double rate_per_second = iterations / total_elapsed; - + Logger::info("Iterations completed: {}", iterations); Logger::info("Total elapsed time: {:.2f} seconds", total_elapsed); Logger::info("Rate: {:.2f} iterations/second", rate_per_second); diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index fd24cb5ed..c2b23bf61 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -282,11 +282,9 @@ On an `m5d.2xlarge` EC2 instance, with `spectator-cpp-2.0` and `github.com/Netflix/spectator-cpp/v2 v2.0.13`, we have observed the following single-threaded performance numbers across a two-minute test window: -* requests/second over `udp` -* requests/second over `unix` +* 113,655.11 requests/second over `udp` +* 132490.97 requests/second over `unix` The benchmark incremented a single counter with two tags in a tight loop, to simulate real-world -tag usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol +tag usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol line was `74` characters in length. - -The Go process CPU usage was ~112% and the `spectatord` process CPU usage was ~62% on this 8 vCPU system, for `udp`. It was ~113% and ~85%, respectively, for `unix`. From dbd75cd1528fca918154a22a90d61881d9b6dca6 Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Sun, 13 Jul 2025 18:38:50 -0500 Subject: [PATCH 08/16] update docs --- docs/spectator/lang/cpp/migrations.md | 20 ++++++++++---------- docs/spectator/lang/cpp/perf-test.md | 2 +- docs/spectator/lang/cpp/usage.md | 23 ++++++++++++----------- 3 files changed, 23 insertions(+), 22 deletions(-) diff --git a/docs/spectator/lang/cpp/migrations.md b/docs/spectator/lang/cpp/migrations.md index 0f4cca21e..33c3f3b51 100644 --- a/docs/spectator/lang/cpp/migrations.md +++ b/docs/spectator/lang/cpp/migrations.md @@ -6,7 +6,7 @@ Version 2.X consists of a major rewrite that greatly simplifies spectator-cpp an #### Writers -`spectator.Registry` now supports 3 different writers. The WriterType is specefied through a WriterConfig object. +`spectator.Registry` now supports 3 different writers. The WriterType is specified through a WriterConfig object. See [Usage > Output Location](usage.md#output-location) for more details. @@ -27,15 +27,15 @@ Note that common tags sourced by [spectatord](https://github.com/Netflix-Skunkwo #### Registry, Config, and Writer Config * `Config` is now created through a constructor which throws error if the passed in parameters are not valid. -* `Config` members are now private. - -### Moved - - - -### Removed - - +* `WriterConfig` now specifies which writer type the thin client uses. +* `WriterConfig` allows line buffering for all writer types. +* `Registry` is instantiated by passing only a `Config` object to it. ### Migration Steps +1. Remove old references to the old spectator library implementation. +2. Utilize the `Config` & `WriterConfig` to initialize the `Registry`. +3. Currently there is not support to collect runtime metrics for the spectator-cpp library. +4. If you need to configure a `Registry` that doesn't emit metrics, for testing purposes, you can +use the `WriterConfig` to configure a `MemoryWriter`. This will emit metrics to a vector so make +sure to clear the vector every so often. diff --git a/docs/spectator/lang/cpp/perf-test.md b/docs/spectator/lang/cpp/perf-test.md index 7ff213e00..ebe2f7f74 100644 --- a/docs/spectator/lang/cpp/perf-test.md +++ b/docs/spectator/lang/cpp/perf-test.md @@ -61,4 +61,4 @@ int main() ## Results -See [Usage > Performance](usage.md#performance). \ No newline at end of file +See [Usage > Performance](usage.md#performance). diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index c2b23bf61..ec1bb278d 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -8,7 +8,7 @@ CPP thin-client [metrics library] for use with [Atlas] and [SpectatorD]. ## Supported CPP Versions -This library currently utilzes C++ 20. +This library currently utilizes C++ 20. ## Installing & Building @@ -58,7 +58,8 @@ int main() Logging uses the `spdlog` library and outputs to standard output by default, with a default log level of `spdlog::level::info`. The `Logger` class is a singleton and provides the `Logger::GetLogger()` function to access the logger instance. To change the log level, call -`Logger::GetLogger()->set_level(spdlog::level::debug);` after the logger has been successfully created. +`Logger::GetLogger()->set_level(spdlog::level::debug);` after the logger has been successfully +created. ## Runtime Metrics @@ -68,14 +69,14 @@ Coming Soon Each metric stored in Atlas is uniquely identified by the combination of the name and the tags associated with it. In `spectator-cpp`, this data is represented with `MeterId` objects, created -by the `Registry`. The `CreateNewId()` method returns new a `MeterId` object, which have extra common -tags applied, and which can be further customized by calling the `WithTag()` and `WithTags()` -methods. Each `MeterId` will create and store a validated subset of the `spectatord` protocol line -to be written for each `Meter`, when it is instantiated. Manipulating the tags with the provided methods - will create new `MeterId` objects. - -Note that **all tag keys and values must be strings.** For example, if you want to keep track of the -number of successful requests, then you must cast integers to strings. The `Id` class will +by the `Registry`. The `CreateNewId()` method returns new a `MeterId` object, which have extra +common tags applied, and which can be further customized by calling the `WithTag()` and +`WithTags()` methods. Each `MeterId` will create and store a validated subset of the `spectatord` +protocol line to be written for each `Meter`, when it is instantiated. Manipulating the tags with +the provided methods will create new `MeterId` objects. + +Note that **all tag keys and values must be strings.** For example, if you want to keep track of +the number of successful requests, then you must cast integers to strings. The `Id` class will validate these values, dropping or changing any that are not valid, and reporting a warning log. {% raw %} @@ -287,4 +288,4 @@ performance numbers across a two-minute test window: The benchmark incremented a single counter with two tags in a tight loop, to simulate real-world tag usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol -line was `74` characters in length. +line was `74` characters in length. \ No newline at end of file From f776bc00a176a577412ef99bb2804bbae3c16d97 Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Sun, 13 Jul 2025 18:39:59 -0500 Subject: [PATCH 09/16] add commas --- docs/spectator/lang/cpp/usage.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index ec1bb278d..f86ffead0 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -284,8 +284,8 @@ On an `m5d.2xlarge` EC2 instance, with `spectator-cpp-2.0` and performance numbers across a two-minute test window: * 113,655.11 requests/second over `udp` -* 132490.97 requests/second over `unix` +* 132,490.97 requests/second over `unix` The benchmark incremented a single counter with two tags in a tight loop, to simulate real-world tag usage, and the rate-per-second observed on the corresponding Atlas graph matched. The protocol -line was `74` characters in length. \ No newline at end of file +line was `74` characters in length. From c173bdfdfe56c5bbcc78d9f9228b1ffacfac0c93 Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Sun, 13 Jul 2025 18:46:09 -0500 Subject: [PATCH 10/16] fix col width --- docs/spectator/lang/cpp/meters/gauge.md | 1 - docs/spectator/lang/cpp/migrations.md | 3 ++- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/spectator/lang/cpp/meters/gauge.md b/docs/spectator/lang/cpp/meters/gauge.md index afb7b8076..924b96e0b 100644 --- a/docs/spectator/lang/cpp/meters/gauge.md +++ b/docs/spectator/lang/cpp/meters/gauge.md @@ -46,5 +46,4 @@ int main() auto serverQueueMeter = r.CreateNewId("server.queueSize"); r.CreateGauge(serverQueueMeter, 120).Set(10); } - ``` diff --git a/docs/spectator/lang/cpp/migrations.md b/docs/spectator/lang/cpp/migrations.md index 33c3f3b51..cefdabbbe 100644 --- a/docs/spectator/lang/cpp/migrations.md +++ b/docs/spectator/lang/cpp/migrations.md @@ -1,6 +1,7 @@ ## Migrating to 2.X -Version 2.X consists of a major rewrite that greatly simplifies spectator-cpp and the process in which it sends metrics to SpectatorD +Version 2.X consists of a major rewrite that greatly simplifies spectator-cpp and the process in +which it sends metrics to SpectatorD ### New From 9775c7b316a1c239d45e97bb1e91b7bc049beaf5 Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Mon, 14 Jul 2025 12:27:02 -0500 Subject: [PATCH 11/16] nit fixes --- docs/spectator/lang/cpp/migrations.md | 8 ++++---- docs/spectator/lang/cpp/usage.md | 6 +++--- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/spectator/lang/cpp/migrations.md b/docs/spectator/lang/cpp/migrations.md index cefdabbbe..2ab24987a 100644 --- a/docs/spectator/lang/cpp/migrations.md +++ b/docs/spectator/lang/cpp/migrations.md @@ -1,7 +1,7 @@ ## Migrating to 2.X Version 2.X consists of a major rewrite that greatly simplifies spectator-cpp and the process in -which it sends metrics to SpectatorD +which it sends metrics to SpectatorD. ### New @@ -27,7 +27,7 @@ Note that common tags sourced by [spectatord](https://github.com/Netflix-Skunkwo #### Registry, Config, and Writer Config -* `Config` is now created through a constructor which throws error if the passed in parameters are not valid. +* `Config` is now created through a constructor which throws an error, if the passed in parameters are not valid. * `WriterConfig` now specifies which writer type the thin client uses. * `WriterConfig` allows line buffering for all writer types. * `Registry` is instantiated by passing only a `Config` object to it. @@ -36,7 +36,7 @@ Note that common tags sourced by [spectatord](https://github.com/Netflix-Skunkwo 1. Remove old references to the old spectator library implementation. 2. Utilize the `Config` & `WriterConfig` to initialize the `Registry`. -3. Currently there is not support to collect runtime metrics for the spectator-cpp library. +3. Currently there is no support for collecting runtime metrics, using the spectator-cpp library. 4. If you need to configure a `Registry` that doesn't emit metrics, for testing purposes, you can -use the `WriterConfig` to configure a `MemoryWriter`. This will emit metrics to a vector so make +use the `WriterConfig` to configure a `MemoryWriter`. This will emit metrics to a vector, so make sure to clear the vector every so often. diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index f86ffead0..95f4f2c09 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -16,7 +16,7 @@ If your project uses CMake, you can easily integrate this library by calling `ad on the root folder. To build the Spectator-CPP thin client independently, follow the Docker container instructions at [Dockerfiles](https://github.com/Netflix/spectator-cpp/tree/main/Dockerfiles). The container provides a minimal build environment with g++-13, python3, and conan. Spectator-CPP -relies on just three external dependencies—spdlog, gtest, and boost—which are managed automatically +relies on just three external dependencies (spdlog, gtest, and boost), which are managed automatically via conan. ## Instrumenting Code @@ -69,7 +69,7 @@ Coming Soon Each metric stored in Atlas is uniquely identified by the combination of the name and the tags associated with it. In `spectator-cpp`, this data is represented with `MeterId` objects, created -by the `Registry`. The `CreateNewId()` method returns new a `MeterId` object, which have extra +by the `Registry`. The `CreateNewId()` method returns new a `MeterId` object, which has extra common tags applied, and which can be further customized by calling the `WithTag()` and `WithTags()` methods. Each `MeterId` will create and store a validated subset of the `spectatord` protocol line to be written for each `Meter`, when it is instantiated. Manipulating the tags with @@ -203,7 +203,7 @@ Registry registry(config); Location can also be set through the environment variable `SPECTATOR_OUTPUT_LOCATION`. If both are set, the environment variable takes precedence over the value passed to the WriterConfig. If either -values provided to the WriterConfig are invalid a runtime exception will be thrown. +values provided to the WriterConfig are invalid, then a runtime exception will be thrown. ## Line Buffer From a93e6212dc217ac3edeeb706e878fb14f5d9106d Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Mon, 14 Jul 2025 12:43:46 -0500 Subject: [PATCH 12/16] Remove runtime metrics --- docs/spectator/lang/cpp/usage.md | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index 95f4f2c09..81e3c8e61 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -61,10 +61,6 @@ level of `spdlog::level::info`. The `Logger` class is a singleton and provides t `Logger::GetLogger()->set_level(spdlog::level::debug);` after the logger has been successfully created. -## Runtime Metrics - -Coming Soon - ## Working with MeterId Objects Each metric stored in Atlas is uniquely identified by the combination of the name and the tags @@ -280,8 +276,8 @@ int main() ## Performance On an `m5d.2xlarge` EC2 instance, with `spectator-cpp-2.0` and -`github.com/Netflix/spectator-cpp/v2 v2.0.13`, we have observed the following single-threaded -performance numbers across a two-minute test window: +`github.com/Netflix/spectator-cpp/v2 v2.0.0`, we have observed the following single-threaded +performance numbers across a two-minute test window (unbuffered scenario): * 113,655.11 requests/second over `udp` * 132,490.97 requests/second over `unix` From d82410af0dfe6a4d4489f72d0eb36eeebe8128cb Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Mon, 14 Jul 2025 13:00:34 -0500 Subject: [PATCH 13/16] remove raw for one code block --- docs/spectator/lang/cpp/usage.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index 81e3c8e61..313b22981 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -141,8 +141,6 @@ WriterConfig(const std::string& type, unsigned int bufferSize); ### Writer Config Examples -{% raw %} - ```cpp /* Default Writer Config Examples */ @@ -166,8 +164,6 @@ std::string unixUrl = std::string(WriterTypes::UnixURL) + "/var/run/custom/socke WriterConfig wConfig(unixUrl, 4096); ``` -{% endraw %} - ### Config Constructor ```cpp From adb8fed3ca0e2188f690775ec49bef5805fab341 Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Mon, 14 Jul 2025 13:22:38 -0500 Subject: [PATCH 14/16] update Meter descriptions --- docs/spectator/lang/cpp/meters/dist-summary.md | 4 +++- docs/spectator/lang/cpp/meters/percentile-dist-summary.md | 4 +++- docs/spectator/lang/cpp/usage.md | 8 +++++--- 3 files changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/spectator/lang/cpp/meters/dist-summary.md b/docs/spectator/lang/cpp/meters/dist-summary.md index ccf75ed57..b0de9154b 100644 --- a/docs/spectator/lang/cpp/meters/dist-summary.md +++ b/docs/spectator/lang/cpp/meters/dist-summary.md @@ -1,6 +1,8 @@ A Distribution Summary is used to track the distribution of events. It is similar to a Timer, but more general, in that the size does not have to be a period of time. For example, a Distribution -Summary could be used to measure the payload sizes of requests hitting a server. +Summary could be used to measure the payload sizes of requests hitting a server. Note that the C++ +implementation of Distribution Summary allows for the recording of floating point values, which the +other thin clients do not allow. Always use base units when recording data, to ensure that the tick labels presented on Atlas graphs are readable. If you are measuring payload size, then use bytes, not kilobytes (or some other unit). diff --git a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md index 2adbb99e0..94fb7d875 100644 --- a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md +++ b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md @@ -2,7 +2,9 @@ The value tracks the distribution of events, with percentile estimates. It is si `PercentileTimer`, but more general, because the size does not have to be a period of time. For example, it can be used to measure the payload sizes of requests hitting a server or the -number of records returned from a query. +number of records returned from a query. Note that the C++ implementation of Percentile Distribution +Summary allows for the recording of floating point values, which the other thin clients do not +allow. In order to maintain the data distribution, they have a higher storage cost, with a worst-case of up to 300X that of a standard Distribution Summary. Be diligent about any additional dimensions diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index 313b22981..e9d43eb55 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -71,9 +71,11 @@ common tags applied, and which can be further customized by calling the `WithTag protocol line to be written for each `Meter`, when it is instantiated. Manipulating the tags with the provided methods will create new `MeterId` objects. -Note that **all tag keys and values must be strings.** For example, if you want to keep track of -the number of successful requests, then you must cast integers to strings. The `Id` class will -validate these values, dropping or changing any that are not valid, and reporting a warning log. +Note that **all tag keys and values must be strings.** Tags are represented as +`std::unordered_map`. For example, to track the number of successful +requests, ensure your tags use string values, such as `{"statusCode": std::to_string(200)}`. The +`MeterId` class validates all provided tag keys and values: if either is empty or contains only +whitespace, it will be dropped, and any invalid characters will be replaced with an underscore. {% raw %} From b729c512ca1115f2f0ac5e8c4e54a62b818d6c96 Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Mon, 14 Jul 2025 13:31:33 -0500 Subject: [PATCH 15/16] update buffering section --- docs/spectator/lang/cpp/usage.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index e9d43eb55..12c95314d 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -235,8 +235,10 @@ This client is stateless, and sends a UDP packet (or unixgram) to `spectatord` e updated. If you are performing high-volume operations, on the order of tens-of-thousands or millions of operations per second, then you should pre-aggregate your metrics and report them at a cadence closer to the `spectatord` publish interval of 5 seconds. This will keep the CPU usage related to -`spectator-go` and `spectatord` low (around 1% or less), as compared to up to 40% for high-volume -scenarios. +`spectator-cpp` and `spectatord` low (around 1% or less), as compared to up to 40% for high-volume +scenarios. If you choose to use the `WriterConfig` with the buffering feature enabled, metrics will +only be sent when the buffer exceeds its size. The buffer is protected by a mutex, allowing +multiple threads to safely write metrics concurrently. ## Writing Tests From cc4513a19a11129b880304d77424cedb7a1dc9fa Mon Sep 17 00:00:00 2001 From: Everett Badeaux Date: Mon, 14 Jul 2025 13:45:39 -0500 Subject: [PATCH 16/16] change writer location --- docs/spectator/lang/cpp/meters/age-gauge.md | 4 ++-- docs/spectator/lang/cpp/meters/counter.md | 4 ++-- docs/spectator/lang/cpp/meters/dist-summary.md | 2 +- docs/spectator/lang/cpp/meters/gauge.md | 4 ++-- docs/spectator/lang/cpp/meters/max-gauge.md | 2 +- docs/spectator/lang/cpp/meters/monotonic-counter-uint.md | 2 +- docs/spectator/lang/cpp/meters/monotonic-counter.md | 2 +- docs/spectator/lang/cpp/meters/percentile-dist-summary.md | 2 +- docs/spectator/lang/cpp/meters/percentile-timer.md | 2 +- docs/spectator/lang/cpp/meters/timer.md | 2 +- docs/spectator/lang/cpp/usage.md | 4 ++-- 11 files changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/spectator/lang/cpp/meters/age-gauge.md b/docs/spectator/lang/cpp/meters/age-gauge.md index f13ba34d6..30872656c 100644 --- a/docs/spectator/lang/cpp/meters/age-gauge.md +++ b/docs/spectator/lang/cpp/meters/age-gauge.md @@ -11,7 +11,7 @@ To set a specific time as the last success: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create an Age Gauge @@ -31,7 +31,7 @@ To set `Now()` as the last success: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create an Age Gauge diff --git a/docs/spectator/lang/cpp/meters/counter.md b/docs/spectator/lang/cpp/meters/counter.md index 75525cfe8..cb4c13434 100644 --- a/docs/spectator/lang/cpp/meters/counter.md +++ b/docs/spectator/lang/cpp/meters/counter.md @@ -11,7 +11,7 @@ Call `Increment()` when an event occurs: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Counter @@ -32,7 +32,7 @@ together: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Counter diff --git a/docs/spectator/lang/cpp/meters/dist-summary.md b/docs/spectator/lang/cpp/meters/dist-summary.md index b0de9154b..4ae31c9e3 100644 --- a/docs/spectator/lang/cpp/meters/dist-summary.md +++ b/docs/spectator/lang/cpp/meters/dist-summary.md @@ -15,7 +15,7 @@ Call `Record()` with a value: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Distribution Summary diff --git a/docs/spectator/lang/cpp/meters/gauge.md b/docs/spectator/lang/cpp/meters/gauge.md index 924b96e0b..bc2c38d3b 100644 --- a/docs/spectator/lang/cpp/meters/gauge.md +++ b/docs/spectator/lang/cpp/meters/gauge.md @@ -13,7 +13,7 @@ Call `Set()` with a value: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Gauge @@ -35,7 +35,7 @@ graphs. A custom TTL may be configured for gauges. SpectatorD enforces a minimum int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Gauge diff --git a/docs/spectator/lang/cpp/meters/max-gauge.md b/docs/spectator/lang/cpp/meters/max-gauge.md index 5712b7ecc..8ad17a9e8 100644 --- a/docs/spectator/lang/cpp/meters/max-gauge.md +++ b/docs/spectator/lang/cpp/meters/max-gauge.md @@ -10,7 +10,7 @@ Call `Set()` with a value: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Max Gauge diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md index 0a772f5be..5284a99e2 100644 --- a/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md +++ b/docs/spectator/lang/cpp/meters/monotonic-counter-uint.md @@ -11,7 +11,7 @@ Call `Set()` when an event occurs: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Monotonic Counter uint64_t diff --git a/docs/spectator/lang/cpp/meters/monotonic-counter.md b/docs/spectator/lang/cpp/meters/monotonic-counter.md index b127102d1..06d3580d1 100644 --- a/docs/spectator/lang/cpp/meters/monotonic-counter.md +++ b/docs/spectator/lang/cpp/meters/monotonic-counter.md @@ -11,7 +11,7 @@ Call `Set()` when an event occurs: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Monotonic Counter diff --git a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md index 94fb7d875..0bc4aae2e 100644 --- a/docs/spectator/lang/cpp/meters/percentile-dist-summary.md +++ b/docs/spectator/lang/cpp/meters/percentile-dist-summary.md @@ -17,7 +17,7 @@ Call `Record()` with a value: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Percentile Distribution Summary diff --git a/docs/spectator/lang/cpp/meters/percentile-timer.md b/docs/spectator/lang/cpp/meters/percentile-timer.md index b28f3b878..ded4221ec 100644 --- a/docs/spectator/lang/cpp/meters/percentile-timer.md +++ b/docs/spectator/lang/cpp/meters/percentile-timer.md @@ -15,7 +15,7 @@ Call `Record()` with a value: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Percentile Timer diff --git a/docs/spectator/lang/cpp/meters/timer.md b/docs/spectator/lang/cpp/meters/timer.md index d3d4e11c7..e770bf6c4 100644 --- a/docs/spectator/lang/cpp/meters/timer.md +++ b/docs/spectator/lang/cpp/meters/timer.md @@ -7,7 +7,7 @@ Call `Record()` with a value: int main() { - auto config = Config(WriterConfig(WriterTypes::Memory)); + auto config = Config(WriterConfig(WriterTypes::UDP)); auto r = Registry(config); // Option 1: Directly create a Timer diff --git a/docs/spectator/lang/cpp/usage.md b/docs/spectator/lang/cpp/usage.md index 12c95314d..5054d6c02 100644 --- a/docs/spectator/lang/cpp/usage.md +++ b/docs/spectator/lang/cpp/usage.md @@ -32,7 +32,7 @@ int main() std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; // Create a config which defines the way you send metrics to SpectatorD - auto config = Config(WriterConfig(WriterTypes::Memory), commonTags); + auto config = Config(WriterConfig(WriterTypes::UDP), commonTags); // Initialize the Registry with the Config (Always required before sending metrics) auto r = Registry(config); @@ -88,7 +88,7 @@ int main() std::unordered_map commonTags{{"platform", "my-platform"}, {"process", "my-process"}}; // Initialize the Registry - auto config = Config(WriterConfig(WriterTypes::Memory), commonTags); + auto config = Config(WriterConfig(WriterTypes::UDP), commonTags); auto registry = Registry(config);