RW 2.0-rc.4 (breaking): Move CT from TimeSeries to Sample; Rename to ST (Start Timestamp)

[bwplotka]

Given the recent movement for Prometheus native support of ST (PROM-60) and plans for delta temporality
(PROM-48) it might be beneficial to make (hopefully) last change to Remote Write 2.0 before stabilizing, so:

• Rename Created Timestamp to Start Timestamp.
• Move CT/ST from TimeSeries to Sample and Histogram messages.
• Clarified optionality (0 value meaning unset).

Rationales (for moving CT/ST to Sample)

The biggest change is the move from CT/ST per TimeSeries to be per Sample. This makes sample a triplet (st, t, v) not a pair (t, v).

Let's go through pros & cons of this change:

Pro

• It logically cleaner (ST relates to sample semantics around timestamps) for both cumulatives and delta one day.
• Future consistency with 1P (first party): PROM-60 for native support of ST, all APIs will logically allow ST per sample. It will be awkward to pass this to per series for RW (and back).
• Consistency with 3P (third party system: Known systems that supports CT/ST store it per sample (Otel, Google).
• Has no practical impact on current performance because Prometheus always sends one sample per timeseries message.
• If, one day, use cases emerge to send more than one sample per TimeSeries with different ST values (e.g. counter reset or delta), it's will much easier to read and write (vs duplicating timeseries message, so essentially creating a new series) and potentially less efficient (duplicate TimeSeries message).

Cons

• If, one day, use cases emerge to send more than one sample per TimeSeries with the same ST values, there will be some overhead of extra, duplicated int64 (there's no diff compression like in chunk).
  • However, this is exactly the same for t,v and we didn't optimize it because RW never had use cases beyond one-sample streams. If we start seeing more use cases around bulk imports using RW with dozens-hundrends of samples per timeSeries, we can do 2.1 that adds chunks or tailored semantics for samples.
• We make a breaking change in an experimental RW (however, the impact is low, see below)

Low Breaking Change Impact

This change is in theory breaking. However, practically we impact should be minimal. See implementation change that will follow. Notice that only receiver part was implemented for CT/ST. Given no sending part was done we expect this feature (ST/CT) not being used, thus breakage impact is minimal. This has been confirmed with early adopters like Mimir (Grafana), Chronosphere, Thanos, Cortex and Google.

See previous discussions and 3 explicit approvals: prometheus/prometheus#17036

Alternatives Considered

A. Don't do move ST. Rejected, as we miss lot's of consistency, cleanup and arguably efficiency factors.
B. Keep ST per series for cumulatives; Add ST per sample. Rejected, as it makes protocol much more complex to explain and implement without benefits. We send only one sample so there's no efficiency different and if we send more eventually cumulatives will change ST causing duplicated timeseries which is not great either. Let's optimize for multiple-sample case when we will have use cases.
C. Add special semantics for ST per sample to only add it when needed (e.g. when it's different). Similar issues as with (B), maybe something to optimize if more multi-sample per series cases emerge.

Additional Spec Improvements

• I updated link to proto.
• Updated links to new compliance tests.
• Update native histogram spec link.

cc @alexgreenbank @cstyan @ArthurSens @aknuds1 @krajorama @bboreham @kgoudeaux

[bwplotka]

I addressed all comments @krajorama

Kind ping @ArthurSens @cstyan

[krajorama]

LGTM

[ArthurSens]

LGTM