[APMAPI-1258] Adjust trace sampling formula

[genesor]

What does this PR do?

Adjust the knuth sampling formula to have the same one across every tracer.

Motivation:

There is slight differences between every tracer in the implementation of this formula. The Ruby one was correct but the modulo was based on the EXTERNAL_MAX_ID value that was wrong (off by one)

Change log entry

Yes. Cosmetic change on the knuth sampling formula to have a clear separation between uint64 modulo in a new const UINT64_MODULO (1<<64) and MAX_EXTERNAL_ID ((1<<64) -1)

Additional Notes:

How to test the change?

• Test added in rate_sampler_spec
• No impact on current tracer behavior

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.74%. Comparing base (56a587c) to head (e39bd7a).

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4616      +/-   ##
==========================================
- Coverage   97.76%   97.74%   -0.02%     
==========================================
  Files        1415     1415              
  Lines       86264    86275      +11     
  Branches     4348     4349       +1     
==========================================
- Hits        84332    84328       -4     
- Misses       1932     1947      +15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[datadog-datadog-prod-us1]

Datadog Report

Branch report: ben.db/APMAPI-1258-adjust-sampling-formula
Commit report: e39bd7a
Test service: dd-trace-rb

✅ 0 Failed, 21254 Passed, 1373 Skipped, 3m 37.28s Total Time

[vpellan]

Left a comment but except the steep error, LGTM !

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-05-01 21:10:27

Comparing candidate commit e39bd7a in PR branch ben.db/APMAPI-1258-adjust-sampling-formula with baseline commit 56a587c in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 31 metrics, 2 unstable metrics.

[p-datadog]

Approved but this PR as far as I can tell does not change the modulo and hence makes no changes to sampling algorithm, it only rejects 1<<64 as a trace id (externally supplied?).

[genesor]

Approved but this PR as far as I can tell does not change the modulo and hence makes no changes to sampling algorithm, it only rejects 1<<64 as a trace id (externally supplied?).

Yes @p-datadog, there is no change in the modulo or the formula itself other than the cosmetic one of separating the modulo and MAX_EXTERNAL_ID.

The old value of MAX_EXTERNAL_ID was an uint65 instead of uint64 (hence the -1)

[p-datadog]

@genesor could you update the changelog entry in the PR description with this please?

[p-datadog]

Does the 1<<64 case whose behavior is changed by the PR need test coverage?

[genesor]

Does the 1<<64 case whose behavior is changed by the PR need test coverage?

@p-datadog I'd say no, this is not an ID we can produce in other languages as it is bound to the limit of uint64 with the value 1<<64 -1