Fix benchmark flakiness #7790

NachoEchevarria · 2025-11-06T16:05:53Z

Summary of changes

This PR addresses benchmark flakiness detected by eliminating static constructors and moving all initialization to [GlobalSetup] methods, which BenchmarkDotNet explicitly excludes from timing measurements.

Benchmarks were experiencing flakiness due to:

Static constructor timing issues: Static constructors run at unpredictable times and their overhead is included in early benchmark measurements
Incorrect static field usage: Using static fields with [GlobalSetup] creates confusion since GlobalSetup runs once per benchmark method, not once per class
Native code warmup: Some benchmarks (especially WAF-related) need some warmup to stabilize native code paths

Why Use Instance Fields for Runtime State?

[GlobalSetup] runs once per benchmark method, not once per class. Using instance fields for runtime state (clients, services, etc.):

Ensures proper isolation - Each benchmark gets a fresh class instance
Prevents state leakage - No shared state between different benchmark methods
Matches BenchmarkDotNet semantics - Clear intent that each benchmark is isolated

Why Use `static readonly` for Test Data?

For pre-computed immutable test data (lambdas, test objects, constant values):

Better compiler optimization - static readonly fields can be inlined and optimized better than instance fields
Single allocation - Test data allocated once, shared across all benchmark runs
Clear intent - Signals that the data never changes and is shared safely

Reason for change

Benchmark variance analysis revealed high Coefficients of Variation (CV) ranging from 5.8% to 151%, indicating unstable and unreliable performance measurements.

Implementation details

Test coverage

Other details

dd-trace-dotnet-ci-bot · 2025-11-06T17:14:17Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (7790) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	74.65 ± (74.73 - 75.44) ms	76.02 ± (76.09 - 76.94) ms	+1.8%	✅⬆️
.NET Framework 4.8 - Bailout
duration	78.86 ± (78.59 - 79.18) ms	81.01 ± (81.08 - 82.07) ms	+2.7%	✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1056.80 ± (1061.21 - 1072.27) ms	1069.86 ± (1072.97 - 1085.66) ms	+1.2%	✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	22.96 ± (22.87 - 23.05) ms	23.36 ± (23.28 - 23.45) ms	+1.8%	✅⬆️
process.time_to_main_ms	87.69 ± (87.30 - 88.09) ms	90.45 ± (89.95 - 90.95) ms	+3.1%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.94 ± (10.94 - 10.94) MB	10.92 ± (10.92 - 10.93) MB	-0.2%	✅
runtime.dotnet.threads.count	12 ± (12 - 12)	12 ± (12 - 12)	+0.0%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	22.90 ± (22.83 - 22.97) ms	23.12 ± (23.05 - 23.19) ms	+1.0%	✅⬆️
process.time_to_main_ms	90.11 ± (89.70 - 90.53) ms	91.04 ± (90.60 - 91.49) ms	+1.0%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.98 ± (10.98 - 10.98) MB	10.95 ± (10.95 - 10.96) MB	-0.2%	✅
runtime.dotnet.threads.count	13 ± (13 - 13)	13 ± (13 - 13)	+0.0%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	220.48 ± (219.02 - 221.93) ms	226.52 ± (225.04 - 228.00) ms	+2.7%	✅⬆️
process.time_to_main_ms	499.02 ± (497.87 - 500.18) ms	512.00 ± (510.65 - 513.35) ms	+2.6%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	47.76 ± (47.74 - 47.79) MB	47.77 ± (47.75 - 47.78) MB	+0.0%	✅⬆️
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.2%	✅⬆️
.NET 6 - Baseline
process.internal_duration_ms	21.65 ± (21.56 - 21.75) ms	22.10 ± (22.01 - 22.18) ms	+2.0%	✅⬆️
process.time_to_main_ms	76.17 ± (75.76 - 76.59) ms	77.87 ± (77.50 - 78.24) ms	+2.2%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.61 ± (10.61 - 10.61) MB	10.64 ± (10.64 - 10.64) MB	+0.3%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 6 - Bailout
process.internal_duration_ms	21.72 ± (21.65 - 21.79) ms	21.87 ± (21.79 - 21.95) ms	+0.7%	✅⬆️
process.time_to_main_ms	77.57 ± (77.24 - 77.89) ms	79.60 ± (79.19 - 80.01) ms	+2.6%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.66 ± (10.66 - 10.66) MB	10.67 ± (10.66 - 10.67) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	213.60 ± (211.40 - 215.79) ms	214.00 ± (212.51 - 215.49) ms	+0.2%	✅⬆️
process.time_to_main_ms	468.00 ± (467.13 - 468.88) ms	478.73 ± (477.36 - 480.09) ms	+2.3%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	48.03 ± (48.00 - 48.05) MB	48.11 ± (48.09 - 48.14) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.0%	✅⬆️
.NET 8 - Baseline
process.internal_duration_ms	20.03 ± (19.96 - 20.10) ms	20.07 ± (19.99 - 20.15) ms	+0.2%	✅⬆️
process.time_to_main_ms	76.19 ± (75.85 - 76.53) ms	76.35 ± (75.91 - 76.79) ms	+0.2%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.66 ± (7.65 - 7.66) MB	7.65 ± (7.65 - 7.66) MB	-0.0%	✅
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 8 - Bailout
process.internal_duration_ms	20.00 ± (19.95 - 20.06) ms	20.10 ± (20.02 - 20.17) ms	+0.5%	✅⬆️
process.time_to_main_ms	77.13 ± (76.84 - 77.42) ms	78.26 ± (77.87 - 78.64) ms	+1.5%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.70 ± (7.70 - 7.71) MB	7.71 ± (7.70 - 7.72) MB	+0.1%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	192.54 ± (191.61 - 193.48) ms	195.24 ± (194.18 - 196.30) ms	+1.4%	✅⬆️
process.time_to_main_ms	456.57 ± (455.67 - 457.48) ms	465.63 ± (464.31 - 466.94) ms	+2.0%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	36.40 ± (36.36 - 36.44) MB	36.54 ± (36.50 - 36.57) MB	+0.4%	✅⬆️
runtime.dotnet.threads.count	27 ± (27 - 27)	27 ± (27 - 27)	+0.1%	✅⬆️

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	192.46 ± (192.43 - 193.26) ms	193.93 ± (193.68 - 194.56) ms	+0.8%	✅⬆️
.NET Framework 4.8 - Bailout
duration	196.34 ± (196.07 - 196.67) ms	197.33 ± (197.14 - 197.96) ms	+0.5%	✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1104.28 ± (1110.04 - 1119.65) ms	1105.45 ± (1111.20 - 1120.81) ms	+0.1%	✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	187.63 ± (187.29 - 187.97) ms	188.39 ± (188.06 - 188.72) ms	+0.4%	✅⬆️
process.time_to_main_ms	80.23 ± (80.04 - 80.42) ms	81.29 ± (81.07 - 81.51) ms	+1.3%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.06 ± (16.03 - 16.09) MB	16.18 ± (16.15 - 16.21) MB	+0.7%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 20)	20 ± (20 - 20)	+0.6%	✅⬆️
.NET Core 3.1 - Bailout
process.internal_duration_ms	187.08 ± (186.77 - 187.38) ms	187.58 ± (187.34 - 187.82) ms	+0.3%	✅⬆️
process.time_to_main_ms	81.72 ± (81.59 - 81.85) ms	82.48 ± (82.34 - 82.61) ms	+0.9%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.15 ± (16.13 - 16.18) MB	16.20 ± (16.17 - 16.23) MB	+0.3%	✅⬆️
runtime.dotnet.threads.count	21 ± (21 - 21)	21 ± (20 - 21)	-0.5%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	401.57 ± (398.56 - 404.58) ms	405.80 ± (402.68 - 408.92) ms	+1.1%	✅⬆️
process.time_to_main_ms	469.82 ± (469.27 - 470.37) ms	475.89 ± (475.07 - 476.71) ms	+1.3%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	58.26 ± (58.12 - 58.41) MB	58.61 ± (58.49 - 58.74) MB	+0.6%	✅⬆️
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 29)	+0.0%	✅⬆️
.NET 6 - Baseline
process.internal_duration_ms	191.33 ± (190.98 - 191.68) ms	192.57 ± (192.17 - 192.97) ms	+0.6%	✅⬆️
process.time_to_main_ms	69.68 ± (69.51 - 69.86) ms	70.62 ± (70.43 - 70.81) ms	+1.3%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.04 ± (15.88 - 16.20) MB	16.23 ± (16.13 - 16.33) MB	+1.2%	✅⬆️
runtime.dotnet.threads.count	18 ± (18 - 18)	18 ± (18 - 19)	+1.7%	✅⬆️
.NET 6 - Bailout
process.internal_duration_ms	190.15 ± (189.93 - 190.37) ms	191.94 ± (191.57 - 192.31) ms	+0.9%	✅⬆️
process.time_to_main_ms	70.37 ± (70.31 - 70.44) ms	71.51 ± (71.41 - 71.62) ms	+1.6%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.02 ± (15.86 - 16.18) MB	16.23 ± (16.10 - 16.35) MB	+1.3%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 19)	20 ± (19 - 20)	+2.0%	✅⬆️
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	418.39 ± (415.12 - 421.66) ms	414.30 ± (411.36 - 417.23) ms	-1.0%	✅
process.time_to_main_ms	442.71 ± (442.09 - 443.34) ms	443.27 ± (442.66 - 443.89) ms	+0.1%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	58.63 ± (58.49 - 58.76) MB	58.79 ± (58.66 - 58.92) MB	+0.3%	✅⬆️
runtime.dotnet.threads.count	29 ± (29 - 30)	30 ± (29 - 30)	+0.1%	✅⬆️
.NET 8 - Baseline
process.internal_duration_ms	190.44 ± (190.10 - 190.78) ms	190.34 ± (190.01 - 190.67) ms	-0.1%	✅
process.time_to_main_ms	69.32 ± (69.15 - 69.48) ms	69.77 ± (69.61 - 69.92) ms	+0.7%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.76 ± (11.74 - 11.79) MB	11.76 ± (11.72 - 11.81) MB	-0.0%	✅
runtime.dotnet.threads.count	18 ± (18 - 18)	18 ± (18 - 18)	-0.4%	✅
.NET 8 - Bailout
process.internal_duration_ms	189.46 ± (189.20 - 189.72) ms	190.39 ± (190.10 - 190.67) ms	+0.5%	✅⬆️
process.time_to_main_ms	70.28 ± (70.17 - 70.39) ms	70.92 ± (70.79 - 71.05) ms	+0.9%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.81 ± (11.76 - 11.87) MB	11.83 ± (11.80 - 11.86) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	19 ± (19 - 19)	19 ± (19 - 19)	+0.8%	✅⬆️
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	370.02 ± (368.60 - 371.44) ms	366.68 ± (365.14 - 368.21) ms	-0.9%	✅
process.time_to_main_ms	433.91 ± (433.08 - 434.74) ms	430.24 ± (429.67 - 430.82) ms	-0.8%	✅
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	47.85 ± (47.82 - 47.88) MB	47.84 ± (47.82 - 47.87) MB	-0.0%	✅
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 29)	+0.6%	✅⬆️

Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts

FakeDbCommand (.NET Framework 4.8)

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7790) - mean (77ms)  : 70, 83
    master - mean (75ms)  : 70, 80

    section Bailout
    This PR (7790) - mean (82ms)  : 74, 89
    master - mean (79ms)  : 74, 83

    section CallTarget+Inlining+NGEN
    This PR (7790) - mean (1,079ms)  : 981, 1177
    master - mean (1,067ms)  : 986, 1147

FakeDbCommand (.NET Core 3.1)

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7790) - mean (121ms)  : 112, 130
    master - mean (118ms)  : 110, 126

    section Bailout
    This PR (7790) - mean (122ms)  : 114, 130
    master - mean (120ms)  : 112, 129

    section CallTarget+Inlining+NGEN
    This PR (7790) - mean (781ms)  : 736, 826
    master - mean (760ms)  : 719, 800

FakeDbCommand (.NET 6)

gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7790) - mean (107ms)  : 99, 115
    master - mean (105ms)  : 99, 110

    section Bailout
    This PR (7790) - mean (109ms)  : 99, 118
    master - mean (106ms)  : 101, 112

    section CallTarget+Inlining+NGEN
    This PR (7790) - mean (726ms)  : 674, 778
    master - mean (713ms)  : 668, 758

FakeDbCommand (.NET 8)

gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7790) - mean (105ms)  : 97, 113
    master - mean (105ms)  : 97, 113

    section Bailout
    This PR (7790) - mean (107ms)  : 99, 114
    master - mean (106ms)  : 101, 111

    section CallTarget+Inlining+NGEN
    This PR (7790) - mean (698ms)  : 665, 731
    master - mean (688ms)  : 660, 716

HttpMessageHandler (.NET Framework 4.8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7790) - mean (194ms)  : 190, 198
    master - mean (193ms)  : 189, 197

    section Bailout
    This PR (7790) - mean (198ms)  : 193, 202
    master - mean (196ms)  : 193, 199

    section CallTarget+Inlining+NGEN
    This PR (7790) - mean (1,116ms)  : 1048, 1184
    master - mean (1,115ms)  : 1045, 1185

HttpMessageHandler (.NET Core 3.1)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7790) - mean (278ms)  : 273, 283
    master - mean (276ms)  : 272, 281

    section Bailout
    This PR (7790) - mean (278ms)  : 274, 283
    master - mean (277ms)  : 274, 280

    section CallTarget+Inlining+NGEN
    This PR (7790) - mean (922ms)  : 876, 968
    master - mean (903ms)  : 851, 955

HttpMessageHandler (.NET 6)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7790) - mean (272ms)  : 265, 278
    master - mean (270ms)  : 264, 276

    section Bailout
    This PR (7790) - mean (272ms)  : 267, 277
    master - mean (268ms)  : 265, 272

    section CallTarget+Inlining+NGEN
    This PR (7790) - mean (893ms)  : 839, 947
    master - mean (893ms)  : 833, 953

HttpMessageHandler (.NET 8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7790) - mean (270ms)  : 266, 275
    master - mean (270ms)  : 265, 274

    section Bailout
    This PR (7790) - mean (271ms)  : 267, 275
    master - mean (269ms)  : 266, 273

    section CallTarget+Inlining+NGEN
    This PR (7790) - mean (828ms)  : 802, 853
    master - mean (835ms)  : 808, 862

NachoEchevarria · 2025-11-07T11:44:27Z

tracer/test/benchmarks/Benchmarks.Trace/README_GUIDELINES.md

@@ -0,0 +1,109 @@
+# Benchmark Guidelines


Guide generated with Claude code.

igoragoli · 2025-11-12T16:00:16Z

Thanks for working on this, @NachoEchevarria!

Seeing errors like these on the CI:

 [ERR] RunBenchmarks: C:\dd-trace-dotnet\tracer\test\benchmarks\Benchmarks.Trace\bin\Release\net6.0\2b73268d-661e-4034-861a-8c6b24014cbc\2b73268d-661e-4034-861a-8c6b24014cbc.notcs(592,33): error CS0103: The name 'GlobalSetup' does not exist in the current context [C:\dd-trace-dotnet\tracer\test\benchmarks\Benchmarks.Trace\bin\Release\net6.0\2b73268d-661e-4034-861a-8c6b24014cbc\BenchmarkDotNet.Autogenerated.csproj]

The benchmarks completes successfully, though.

Because of a rate limiting issue, the script failed to run the PR commenter¹ and failed to upload results to the BP UI (which was done after the PR comment). I'm rue-running the benchmarks so we can get results on the BP UI and I can run the /flakiness endpoint.

That said, looks good to me, excited to see the results.

I took note of that for our team, this shouldn't happen. ↩

pr-commenter · 2025-11-12T16:43:17Z

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7790 compared to master:

3 benchmarks are faster, with geometric mean 1.640
7 benchmarks are slower, with geometric mean 1.439
5 benchmarks have fewer allocations
13 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`StartStopWithChild`	net6.0	10.7μs	53.2ns	219ns	0	0	5.51 KB
master	`StartStopWithChild`	netcoreapp3.1	13.9μs	59.4ns	230ns	0	0	5.7 KB
master	`StartStopWithChild`	net472	22.5μs	119ns	653ns	0.963	0.214	6.1 KB
#7790	`StartStopWithChild`	net6.0	10.2μs	55.4ns	298ns	0	0	5.5 KB
#7790	`StartStopWithChild`	netcoreapp3.1	13.4μs	70.9ns	354ns	0	0	5.71 KB
#7790	`StartStopWithChild`	net472	22.3μs	126ns	857ns	0.96	0.213	6.1 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	934μs	170ns	659ns	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.04ms	280ns	1.09μs	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	1.19ms	240ns	928ns	3.31 KB
#7790	`WriteAndFlushEnrichedTraces`	net6.0	946μs	47.5ns	178ns	2.71 KB
#7790	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.02ms	147ns	531ns	2.7 KB
#7790	`WriteAndFlushEnrichedTraces`	net472	1.23ms	51ns	198ns	3.31 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`AllCycleSimpleBody`	net6.0	1.06μs	6ns	34ns	0	1.22 KB
master	`AllCycleSimpleBody`	netcoreapp3.1	1.44μs	0.837ns	3.24ns	0	1.2 KB
master	`AllCycleSimpleBody`	net472	1.02μs	0.193ns	0.722ns	0.193	1.23 KB
master	`AllCycleMoreComplexBody`	net6.0	7.21μs	35.8ns	160ns	0	4.72 KB
master	`AllCycleMoreComplexBody`	netcoreapp3.1	9μs	36.7ns	137ns	0	4.62 KB
master	`AllCycleMoreComplexBody`	net472	7.69μs	5.41ns	20.9ns	0.731	4.74 KB
master	`ObjectExtractorSimpleBody`	net6.0	319ns	1.72ns	9.7ns	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	385ns	2.18ns	14.1ns	0	272 B
master	`ObjectExtractorSimpleBody`	net472	297ns	0.0397ns	0.148ns	0.0439	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	6.28μs	31.9ns	153ns	0	3.79 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.83μs	2.59ns	9.67ns	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	6.69μs	4.12ns	16ns	0.601	3.8 KB
#7790	`AllCycleSimpleBody`	net6.0	1.09μs	6.13ns	41.6ns	0	1.22 KB
#7790	`AllCycleSimpleBody`	netcoreapp3.1	1.41μs	7.26ns	34.8ns	0	1.2 KB
#7790	`AllCycleSimpleBody`	net472	1.02μs	0.632ns	2.45ns	0.194	1.23 KB
#7790	`AllCycleMoreComplexBody`	net6.0	7.17μs	35.2ns	158ns	0	4.72 KB
#7790	`AllCycleMoreComplexBody`	netcoreapp3.1	8.72μs	46.6ns	251ns	0	4.62 KB
#7790	`AllCycleMoreComplexBody`	net472	7.64μs	2.94ns	11ns	0.728	4.74 KB
#7790	`ObjectExtractorSimpleBody`	net6.0	330ns	1.83ns	12.4ns	0	280 B
#7790	`ObjectExtractorSimpleBody`	netcoreapp3.1	398ns	2.15ns	11.6ns	0	272 B
#7790	`ObjectExtractorSimpleBody`	net472	301ns	0.138ns	0.534ns	0.044	281 B
#7790	`ObjectExtractorMoreComplexBody`	net6.0	6.38μs	32.3ns	155ns	0	3.78 KB
#7790	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.88μs	29.3ns	109ns	0	3.69 KB
#7790	`ObjectExtractorMoreComplexBody`	net472	6.72μs	3.07ns	11.1ns	0.572	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EncodeArgs`	net6.0	76.8μs	186ns	695ns	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	96.8μs	296ns	1.15μs	0	32.4 KB
master	`EncodeArgs`	net472	109μs	7.23ns	27.1ns	4.91	32.51 KB
master	`EncodeLegacyArgs`	net6.0	148μs	56.9ns	213ns	0	2.15 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	199μs	141ns	527ns	0	2.14 KB
master	`EncodeLegacyArgs`	net472	264μs	19.6ns	70.7ns	0	2.16 KB
#7790	`EncodeArgs`	net6.0	76.9μs	20.2ns	75.7ns	0	32.4 KB
#7790	`EncodeArgs`	netcoreapp3.1	97.7μs	234ns	906ns	0	32.4 KB
#7790	`EncodeArgs`	net472	110μs	24.9ns	93.1ns	4.96	32.51 KB
#7790	`EncodeLegacyArgs`	net6.0	143μs	55.3ns	207ns	0	2.15 KB
#7790	`EncodeLegacyArgs`	netcoreapp3.1	199μs	520ns	2.01μs	0	2.14 KB
#7790	`EncodeLegacyArgs`	net472	264μs	22.5ns	87.1ns	0	2.17 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7790

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑net472	1.192	310,066.46	369,700.00
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑net472	1.158	429,711.67	497,500.00

Faster 🎉 in #7790

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑netcoreapp3.1	2.096	720,470.98	343,750.00
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑netcoreapp3.1	1.842	853,429.64	463,400.00

More allocations ⚠️ in #7790

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑net6.0	2.24 KB	2.83 KB	590 B	26.32%
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑net6.0	4.55 KB	5.14 KB	591 B	12.98%
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑netcoreapp3.1	2.22 KB	2.32 KB	104 B	4.69%
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑netcoreapp3.1	4.48 KB	4.58 KB	104 B	2.32%

Fewer allocations 🎉 in #7790

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑net472	4.66 KB	0 b	-4.66 KB	-100.00%
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑net472	2.29 KB	0 b	-2.29 KB	-100.00%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`RunWafRealisticBenchmark`	net6.0	396μs	119ns	460ns	4.55 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	809μs	13.4μs	134μs	4.48 KB
master	`RunWafRealisticBenchmark`	net472	430μs	84.7ns	328ns	4.66 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	289μs	22.3ns	83.3ns	2.24 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	671μs	13.5μs	134μs	2.22 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	310μs	37.6ns	146ns	2.29 KB
#7790	`RunWafRealisticBenchmark`	net6.0	432μs	1.39μs	5.02μs	5.14 KB
#7790	`RunWafRealisticBenchmark`	netcoreapp3.1	472μs	2.94μs	28μs	4.58 KB
#7790	`RunWafRealisticBenchmark`	net472	498μs	279ns	1.01μs	0 b
#7790	`RunWafRealisticBenchmarkWithAttack`	net6.0	319μs	1.09μs	4.07μs	2.83 KB
#7790	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	352μs	2.48μs	23.8μs	2.32 KB
#7790	`RunWafRealisticBenchmarkWithAttack`	net472	370μs	327ns	1.18μs	0 b

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`SendRequest`	net6.0	61.3μs	41.5ns	161ns	14.52 KB
master	`SendRequest`	netcoreapp3.1	71.6μs	63.2ns	245ns	17.42 KB
master	`SendRequest`	net472	0.00307ns	0.00157ns	0.00609ns	0 b
#7790	`SendRequest`	net6.0	61.4μs	30.4ns	118ns	14.52 KB
#7790	`SendRequest`	netcoreapp3.1	71.6μs	75.2ns	271ns	17.42 KB
#7790	`SendRequest`	net472	0.165ns	0.00293ns	0.0114ns	0 b

Benchmarks.Trace.CharSliceBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7790

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑netcoreapp3.1	2.133	873,436.16	1,862,700.00
Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice‑netcoreapp3.1	1.856	2,146,682.03	3,984,300.00
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑netcoreapp3.1	1.645	1,693,476.25	2,786,000.00
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0	1.236	816,465.82	1,009,200.00

More allocations ⚠️ in #7790

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0	2 B	640 B	638 B	31,900.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑netcoreapp3.1	1 B	104 B	103 B	10,300.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑netcoreapp3.1	1 B	104 B	103 B	10,300.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0	7 B	640 B	633 B	9,042.86%
Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice‑net472	641.95 KB	647.17 KB	5.21 KB	0.81%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`OriginalCharSlice`	net6.0	1.91ms	532ns	1.99μs	0	640.01 KB
master	`OriginalCharSlice`	netcoreapp3.1	2.14ms	7.49μs	29μs	0	640 KB
master	`OriginalCharSlice`	net472	2.62ms	881ns	3.41μs	100	641.95 KB
master	`OptimizedCharSlice`	net6.0	1.35ms	2.73μs	10.6μs	0	7 B
master	`OptimizedCharSlice`	netcoreapp3.1	1.69ms	297ns	1.15μs	0	1 B
master	`OptimizedCharSlice`	net472	1.94ms	170ns	638ns	0	0 b
master	`OptimizedCharSliceWithPool`	net6.0	816μs	26.3ns	102ns	0	2 B
master	`OptimizedCharSliceWithPool`	netcoreapp3.1	873μs	52.2ns	195ns	0	1 B
master	`OptimizedCharSliceWithPool`	net472	1.14ms	113ns	439ns	0	0 b
#7790	`OriginalCharSlice`	net6.0	1.91ms	3.6μs	14μs	0	640.64 KB
#7790	`OriginalCharSlice`	netcoreapp3.1	3.98ms	1.28μs	4.42μs	0	640.1 KB
#7790	`OriginalCharSlice`	net472	2.63ms	951ns	3.68μs	0	647.17 KB
#7790	`OptimizedCharSlice`	net6.0	1.43ms	1.67μs	6.01μs	0	640 B
#7790	`OptimizedCharSlice`	netcoreapp3.1	2.79ms	1.45μs	5.62μs	0	104 B
#7790	`OptimizedCharSlice`	net472	1.9ms	521ns	2.02μs	0	0 b
#7790	`OptimizedCharSliceWithPool`	net6.0	1.01ms	800ns	2.88μs	0	640 B
#7790	`OptimizedCharSliceWithPool`	netcoreapp3.1	1.86ms	1.72μs	6.67μs	0	104 B
#7790	`OptimizedCharSliceWithPool`	net472	1.14ms	806ns	3.12μs	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #7790

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472	1.143	1,028,238.54	899,690.21

More allocations ⚠️ in #7790

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472	55.94 KB	56.36 KB	420 B	0.75%

Fewer allocations 🎉 in #7790

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1	42.75 KB	41.87 KB	-878 B	-2.05%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	719μs	937ns	3.51μs	0	41.78 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	748μs	2.2μs	8.54μs	0	42.75 KB
master	`WriteAndFlushEnrichedTraces`	net472	1.03ms	5.08μs	21.6μs	5.21	55.94 KB
#7790	`WriteAndFlushEnrichedTraces`	net6.0	699μs	955ns	3.7μs	0	41.6 KB
#7790	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	729μs	3.9μs	21μs	0	41.87 KB
#7790	`WriteAndFlushEnrichedTraces`	net472	906μs	3.41μs	12.7μs	8.33	56.36 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.89μs	9.71ns	47.6ns	0	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	2.62μs	9.75ns	37.8ns	0	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.86μs	3.4ns	13.2ns	0.156	0.0142	987 B
#7790	`ExecuteNonQuery`	net6.0	1.91μs	7.86ns	30.4ns	0	0	1.02 KB
#7790	`ExecuteNonQuery`	netcoreapp3.1	2.71μs	8.25ns	28.6ns	0	0	1.02 KB
#7790	`ExecuteNonQuery`	net472	2.73μs	2.04ns	7.89ns	0.15	0	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.7μs	7.41ns	27.7ns	0	1.03 KB
master	`CallElasticsearch`	netcoreapp3.1	2.23μs	7.38ns	26.6ns	0	1.03 KB
master	`CallElasticsearch`	net472	3.56μs	3.18ns	12.3ns	0.161	1.04 KB
master	`CallElasticsearchAsync`	net6.0	1.8μs	1.26ns	4.56ns	0	1.01 KB
master	`CallElasticsearchAsync`	netcoreapp3.1	2.32μs	11.2ns	46.2ns	0	1.08 KB
master	`CallElasticsearchAsync`	net472	3.84μs	4.65ns	18ns	0.173	1.1 KB
#7790	`CallElasticsearch`	net6.0	1.72μs	8.83ns	43.3ns	0	1.03 KB
#7790	`CallElasticsearch`	netcoreapp3.1	2.24μs	9.93ns	37.1ns	0	1.03 KB
#7790	`CallElasticsearch`	net472	3.57μs	1.82ns	7.05ns	0.161	1.04 KB
#7790	`CallElasticsearchAsync`	net6.0	1.84μs	6.89ns	26.7ns	0	1.01 KB
#7790	`CallElasticsearchAsync`	netcoreapp3.1	2.45μs	5.24ns	20.3ns	0	1.08 KB
#7790	`CallElasticsearchAsync`	net472	3.86μs	4.04ns	15.7ns	0.174	1.1 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.85μs	5.4ns	20.9ns	0	952 B
master	`ExecuteAsync`	netcoreapp3.1	2.5μs	3.77ns	14.6ns	0	952 B
master	`ExecuteAsync`	net472	2.52μs	0.907ns	3.51ns	0.138	915 B
#7790	`ExecuteAsync`	net6.0	1.9μs	9.72ns	45.6ns	0	952 B
#7790	`ExecuteAsync`	netcoreapp3.1	2.37μs	9.96ns	38.6ns	0	952 B
#7790	`ExecuteAsync`	net472	2.65μs	2.37ns	9.2ns	0.132	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	7μs	4.77ns	17.8ns	0	2.36 KB
master	`SendAsync`	netcoreapp3.1	9.15μs	26.8ns	104ns	0	2.9 KB
master	`SendAsync`	net472	12μs	6.39ns	23.9ns	0.481	3.18 KB
#7790	`SendAsync`	net6.0	6.69μs	6.13ns	23.7ns	0	2.36 KB
#7790	`SendAsync`	netcoreapp3.1	8.48μs	34.9ns	135ns	0	2.9 KB
#7790	`SendAsync`	net472	12.4μs	10.9ns	42.1ns	0.494	3.18 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7790

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	1.152	458,750.00	528,300.00

More allocations ⚠️ in #7790

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	258.88 KB	277.25 KB	18.37 KB	7.10%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	258.66 KB	273.63 KB	14.98 KB	5.79%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0	43.44 KB	43.86 KB	416 B	0.96%

Fewer allocations 🎉 in #7790

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1	43.19 KB	42.87 KB	-320 B	-0.74%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472	65.54 KB	57.34 KB	-8.19 KB	-12.50%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	45.5μs	252ns	2.09μs	43.44 KB
master	`StringConcatBenchmark`	netcoreapp3.1	50μs	290ns	2.28μs	43.19 KB
master	`StringConcatBenchmark`	net472	58.2μs	169ns	632ns	65.54 KB
master	`StringConcatAspectBenchmark`	net6.0	459μs	1.55μs	6.22μs	258.88 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	523μs	2.54μs	10.8μs	258.66 KB
master	`StringConcatAspectBenchmark`	net472	411μs	2.07μs	10.5μs	278.53 KB
#7790	`StringConcatBenchmark`	net6.0	41.3μs	144ns	520ns	43.86 KB
#7790	`StringConcatBenchmark`	netcoreapp3.1	50.8μs	437ns	4.1μs	42.87 KB
#7790	`StringConcatBenchmark`	net472	57.6μs	279ns	1.18μs	57.34 KB
#7790	`StringConcatAspectBenchmark`	net6.0	531μs	1.9μs	6.84μs	277.25 KB
#7790	`StringConcatAspectBenchmark`	netcoreapp3.1	497μs	1.06μs	3.82μs	273.63 KB
#7790	`StringConcatAspectBenchmark`	net472	404μs	2.22μs	17.2μs	278.53 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.67μs	13.7ns	61.1ns	0	1.7 KB
master	`EnrichedLog`	netcoreapp3.1	3.58μs	9.86ns	38.2ns	0	1.7 KB
master	`EnrichedLog`	net472	3.97μs	3.72ns	14.4ns	0.257	1.64 KB
#7790	`EnrichedLog`	net6.0	2.66μs	10.7ns	38.7ns	0	1.7 KB
#7790	`EnrichedLog`	netcoreapp3.1	3.58μs	18.3ns	82ns	0	1.7 KB
#7790	`EnrichedLog`	net472	3.97μs	4.06ns	15.7ns	0.259	1.64 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`EnrichedLog`	net6.0	122μs	72.1ns	260ns	4.31 KB
master	`EnrichedLog`	netcoreapp3.1	128μs	72.6ns	262ns	4.31 KB
master	`EnrichedLog`	net472	166μs	232ns	897ns	4.52 KB
#7790	`EnrichedLog`	net6.0	124μs	97.1ns	376ns	4.31 KB
#7790	`EnrichedLog`	netcoreapp3.1	130μs	274ns	987ns	4.31 KB
#7790	`EnrichedLog`	net472	166μs	114ns	427ns	4.52 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	5μs	23.5ns	97ns	0	2.26 KB
master	`EnrichedLog`	netcoreapp3.1	6.9μs	28.4ns	110ns	0	2.26 KB
master	`EnrichedLog`	net472	7.58μs	4.68ns	17.5ns	0.301	2.08 KB
#7790	`EnrichedLog`	net6.0	4.93μs	17.1ns	66.1ns	0	2.26 KB
#7790	`EnrichedLog`	netcoreapp3.1	6.9μs	31.8ns	123ns	0	2.26 KB
#7790	`EnrichedLog`	net472	7.49μs	5.47ns	21.2ns	0.299	2.08 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	2.01μs	1.23ns	4.61ns	0	1.2 KB
master	`SendReceive`	netcoreapp3.1	2.52μs	12ns	48.2ns	0	1.2 KB
master	`SendReceive`	net472	3.19μs	5.06ns	19.6ns	0.189	1.2 KB
#7790	`SendReceive`	net6.0	2μs	2.73ns	10.6ns	0	1.2 KB
#7790	`SendReceive`	netcoreapp3.1	2.6μs	3.75ns	14.5ns	0	1.2 KB
#7790	`SendReceive`	net472	3.03μs	4.29ns	15.5ns	0.18	1.2 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.31μs	13.3ns	49.9ns	0	1.58 KB
master	`EnrichedLog`	netcoreapp3.1	5.59μs	5.7ns	22.1ns	0	1.63 KB
master	`EnrichedLog`	net472	6.92μs	6.43ns	24.1ns	0.311	2.03 KB
#7790	`EnrichedLog`	net6.0	4.39μs	2.35ns	9.11ns	0	1.58 KB
#7790	`EnrichedLog`	netcoreapp3.1	5.71μs	12.8ns	49.7ns	0	1.63 KB
#7790	`EnrichedLog`	net472	6.85μs	6.24ns	23.4ns	0.308	2.03 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	788ns	4.15ns	20.3ns	0	576 B
master	`StartFinishSpan`	netcoreapp3.1	973ns	4.79ns	19.7ns	0	576 B
master	`StartFinishSpan`	net472	942ns	0.772ns	2.99ns	0.09	578 B
master	`StartFinishScope`	net6.0	930ns	4.96ns	24.8ns	0	696 B
master	`StartFinishScope`	netcoreapp3.1	1.17μs	6.38ns	36.1ns	0	696 B
master	`StartFinishScope`	net472	1.18μs	0.919ns	3.44ns	0.1	658 B
#7790	`StartFinishSpan`	net6.0	780ns	0.126ns	0.472ns	0	576 B
#7790	`StartFinishSpan`	netcoreapp3.1	982ns	4.64ns	18.6ns	0	576 B
#7790	`StartFinishSpan`	net472	912ns	1.22ns	4.71ns	0.0874	578 B
#7790	`StartFinishScope`	net6.0	950ns	4.61ns	18.5ns	0	696 B
#7790	`StartFinishScope`	netcoreapp3.1	1.2μs	6.26ns	30ns	0	696 B
#7790	`StartFinishScope`	net472	1.1μs	0.234ns	0.906ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	1.08μs	5.6ns	22.4ns	0	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	1.49μs	5.97ns	23.1ns	0	696 B
master	`RunOnMethodBegin`	net472	1.45μs	0.655ns	2.45ns	0.102	658 B
#7790	`RunOnMethodBegin`	net6.0	1.06μs	0.598ns	2.32ns	0	696 B
#7790	`RunOnMethodBegin`	netcoreapp3.1	1.41μs	5.16ns	20ns	0	696 B
#7790	`RunOnMethodBegin`	net472	1.45μs	1.06ns	4.11ns	0.101	658 B

NachoEchevarria · 2025-11-12T17:38:49Z

Thanks for working on this, @NachoEchevarria!

Seeing errors like these on the CI:
 [ERR] RunBenchmarks: C:\dd-trace-dotnet\tracer\test\benchmarks\Benchmarks.Trace\bin\Release\net6.0\2b73268d-661e-4034-861a-8c6b24014cbc\2b73268d-661e-4034-861a-8c6b24014cbc.notcs(592,33): error CS0103: The name 'GlobalSetup' does not exist in the current context [C:\dd-trace-dotnet\tracer\test\benchmarks\Benchmarks.Trace\bin\Release\net6.0\2b73268d-661e-4034-861a-8c6b24014cbc\BenchmarkDotNet.Autogenerated.csproj]
The benchmarks completes successfully, though.

Because of a rate limiting issue, the script failed to run the PR commenter1 and failed to upload results to the BP UI (which was done after the PR comment). I'm rue-running the benchmarks so we can get results on the BP UI and I can run the /flakiness endpoint.

That said, looks good to me, excited to see the results.

Footnotes

I took note of that for our team, this shouldn't happen. ↩

Thanks! I will check the cause of the errors.

igoragoli · 2025-11-13T11:14:46Z

I ran the flakiness check again for 0f5d7f6, looks like there were some big improvements!

E.g., Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net6.0 went from 0.478733 to 0.004474.

Full check: https://gist.github.com/igoragoli/806c821ba255e1a8fcad6538307ae323

igoragoli · 2025-11-14T18:02:14Z

I simplified the /flakiness endpoint, so tests are slightly different.

Full check comparing this branch at 82bfad against master at 8603e6d: https://docs.google.com/spreadsheets/d/1LKKYSYefti9l4AVA1imZSssPes0aDuqkSUD7zfxlONw/edit?usp=sharing

igoragoli

Tests are a bit inconclusive (there's still a bit flakiness and many benchmarks on this branch weren't run), but approving since changes look good to me!

andrewlock

Thanks! I have some questions about the remaining statics though - I can't see how we justify keeping them when ditching all the others? 😄

tracer/test/benchmarks/Benchmarks.Trace/ActivityBenchmark.cs

tracer/test/benchmarks/Benchmarks.Trace/Asm/AppSecEncodeBenchmark.cs

tracer/test/benchmarks/Benchmarks.Trace/Asm/AppSecWafBenchmark.cs

tracer/test/benchmarks/Benchmarks.Trace/README_GUIDELINES.md

daniel-romano-DD

Thanks a lot

NachoEchevarria added 2 commits November 6, 2025 16:47

Rely on [GlobalSetup] instead of constructors

3e55742

Benchmark guidelines

90e6e7c

github-actions bot added the area:tests unit tests, integration tests label Nov 6, 2025

Move to globalSetup

0f5d7f6

NachoEchevarria commented Nov 7, 2025

View reviewed changes

igoragoli self-requested a review November 12, 2025 13:21

NachoEchevarria added 3 commits November 13, 2025 11:20

Fix GlobalSetup Error

9682e42

Fix StringAspectBenchmark

9ff09f3

Fix possible null exception

58412b0

This comment has been minimized.

Sign in to view

Try to improve flakyness in flaky tests

82bfad7

igoragoli approved these changes Nov 14, 2025

View reviewed changes

NachoEchevarria changed the title ~~Fix benchmark flakyness~~ Fix benchmark flakiness Nov 17, 2025

NachoEchevarria added 9 commits November 17, 2025 18:04

make static some instance fields

9418a8e

Restore IterationSetup

d129961

Remove DisableTieredCompilation attribute

6093b94

update guide

2912350

update guide

fa04ed0

static -> instance

2234992

Update other benchmarks

f97eebb

refactor security to static readonly

ee3f50b

static for mocks

b428bc0

NachoEchevarria marked this pull request as ready for review November 24, 2025 11:37

NachoEchevarria requested a review from a team as a code owner November 24, 2025 11:37

andrewlock approved these changes Nov 24, 2025

View reviewed changes

NachoEchevarria added 3 commits November 24, 2025 15:01

Use isntance fields instead of static readonly

29d2495

Remove warmup calls

c8e44d3

Fix function pointer errors

1c90ef9

daniel-romano-DD approved these changes Nov 25, 2025

View reviewed changes

NachoEchevarria merged commit 67c35ae into master Nov 25, 2025
153 checks passed

NachoEchevarria deleted the nacho/FixBenchmarkFlakyness branch November 25, 2025 17:08

github-actions bot added this to the vNext-v3 milestone Nov 25, 2025

Fix benchmark flakiness #7790

Fix benchmark flakiness #7790

Uh oh!

Conversation

NachoEchevarria commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Why Use Instance Fields for Runtime State?

Why Use static readonly for Test Data?

Reason for change

Implementation details

Test coverage

Other details

Uh oh!

dd-trace-dotnet-ci-bot bot commented Nov 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

FakeDbCommand

HttpMessageHandler

Uh oh!

NachoEchevarria Nov 7, 2025

Choose a reason for hiding this comment

Uh oh!

igoragoli commented Nov 12, 2025

Footnotes

Uh oh!

pr-commenter bot commented Nov 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #7790

Faster 🎉 in #7790

More allocations ⚠️ in #7790

Fewer allocations 🎉 in #7790

Raw results

Raw results

Slower ⚠️ in #7790

More allocations ⚠️ in #7790

Raw results

Faster 🎉 in #7790

More allocations ⚠️ in #7790

Fewer allocations 🎉 in #7790

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #7790

More allocations ⚠️ in #7790

Fewer allocations 🎉 in #7790

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Uh oh!

NachoEchevarria commented Nov 12, 2025

Footnotes

Uh oh!

igoragoli commented Nov 13, 2025

Uh oh!

This comment has been minimized.

igoragoli commented Nov 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

igoragoli left a comment

Choose a reason for hiding this comment

Uh oh!

andrewlock left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

NachoEchevarria commented Nov 6, 2025 •

edited

Loading

Why Use `static readonly` for Test Data?

dd-trace-dotnet-ci-bot bot commented Nov 6, 2025 •

edited

Loading

pr-commenter bot commented Nov 12, 2025 •

edited

Loading

igoragoli commented Nov 14, 2025 •

edited

Loading