Skip to content

Fix Asm benchmark and stop-reinitializing tracer #7750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Nov 3, 2025
Merged

Fix Asm benchmark and stop-reinitializing tracer #7750

merged 1 commit into from
Nov 3, 2025

Conversation

@andrewlock
Copy link
Member

@andrewlock andrewlock commented Nov 3, 2025

Summary of changes

  • Fix ASM benchmarks by not returning null from EmptyDatadogTracer.PerTraceSettings
  • Stop re-initializing TracerSettings for every execution of the ASM benchmark

Reason for change

The ASM benchmarks have been broken since #7543 (I think). This is because the EmptyDatadogTracer stub used in the benchmark returns null from PerTraceSettings (which can't happen in practice).

Additionally, noticed that the benchmark is repeatedly creating a new TracerSettings object with every execution, which will add noise and be much more expensive than real life.

Implementation details

  • Ensure EmptyDatadogTracer.PerTraceSettings returns a "real" value
  • Stop rebuilding TracerSettings with every execution

Test coverage

This is the test

@andrewlock andrewlock requested a review from a team as a code owner November 3, 2025 10:27
@andrewlock andrewlock added area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc type:flake-fix labels Nov 3, 2025
@github-actions github-actions bot added the area:tests unit tests, integration tests label Nov 3, 2025
andrewlock
andrewlock commented Nov 3, 2025
View reviewed changes
tracer/test/benchmarks/Benchmarks.Trace/Asm/EmptyDatadogTracer.cs
Comment on lines -26 to +38
PerTraceSettings IDatadogTracer.PerTraceSettings => null;
PerTraceSettings IDatadogTracer.PerTraceSettings => _perTraceSettings;
Copy link
Member Author

@andrewlock andrewlock Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to move this to an explicit file, because the new PerTraceSettings instance needs to reference the Settings, and you can't do that here plus because they're explicitly implemented interfaces (they have to be) you can initialize them in the constructor 😅

chatgpt-codex-connector[bot]
chatgpt-codex-connector bot reviewed Nov 3, 2025
View reviewed changes
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

tracer/test/benchmarks/Benchmarks.Trace/Asm/EmptyDatadogTracer.cs
Comment on lines +27 to +30
Settings = new(new NullConfigurationSource());
_gitMetadata = new NullGitMetadataProvider();
_perTraceSettings = new PerTraceSettings(null, null, null, Settings.InitialMutableSettings);
}
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Provide real NamingSchema for benchmark tracer

The newly created benchmark stub still initializes PerTraceSettings with null for the NamingSchema. PerTraceSettings dereferences Schema in GetServiceName and other callers (e.g., tracer.CurrentTraceSettings.Schema.Messaging) assume it is non‑null. As soon as any benchmarked code asks the tracer for a service name, this stub will throw NullReferenceException, so the regression the commit aims to fix will persist. Construct a real NamingSchema from the settings (the Security unit test stub shows an example) instead of passing null.

Useful? React with 👍 / 👎.

Copy link
Member Author

@andrewlock andrewlock Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was lazy, because there's a bunch of refactoring that will come up around this shortly. I don't believe the benchmark calls this, so I don't think it's required

Copy link
Member Author

@andrewlock andrewlock Nov 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed - no more errors in the benchmarks

@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Nov 3, 2025

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7750) - mean (72ms)  : 71, 73
     .   : milestone, 72,
    master - mean (72ms)  : 71, 73
     .   : milestone, 72,

    section Baseline
    This PR (7750) - mean (68ms)  : 67, 70
     .   : milestone, 68,
    master - mean (68ms)  : 67, 70
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7750) - mean (1,052ms)  : 996, 1107
     .   : milestone, 1052,
    master - mean (1,045ms)  : 1002, 1087
     .   : milestone, 1045,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7750) - mean (107ms)  : 106, 109
     .   : milestone, 107,
    master - mean (107ms)  : 106, 108
     .   : milestone, 107,

    section Baseline
    This PR (7750) - mean (107ms)  : 103, 110
     .   : milestone, 107,
    master - mean (106ms)  : 102, 109
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (7750) - mean (754ms)  : 714, 794
     .   : milestone, 754,
    master - mean (747ms)  : 724, 771
     .   : milestone, 747,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7750) - mean (95ms)  : 94, 95
     .   : milestone, 95,
    master - mean (94ms)  : 93, 95
     .   : milestone, 94,

    section Baseline
    This PR (7750) - mean (94ms)  : 92, 96
     .   : milestone, 94,
    master - mean (93ms)  : 91, 95
     .   : milestone, 93,

    section CallTarget+Inlining+NGEN
    This PR (7750) - mean (711ms)  : 675, 746
     .   : milestone, 711,
    master - mean (711ms)  : 668, 753
     .   : milestone, 711,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7750) - mean (93ms)  : 92, 95
     .   : milestone, 93,
    master - mean (93ms)  : 91, 94
     .   : milestone, 93,

    section Baseline
    This PR (7750) - mean (92ms)  : 90, 95
     .   : milestone, 92,
    master - mean (92ms)  : 89, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (7750) - mean (665ms)  : 648, 682
     .   : milestone, 665,
    master - mean (665ms)  : 648, 683
     .   : milestone, 665,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7750) - mean (193ms)  : 188, 198
     .   : milestone, 193,
    master - mean (194ms)  : 190, 198
     .   : milestone, 194,

    section Baseline
    This PR (7750) - mean (190ms)  : 183, 196
     .   : milestone, 190,
    master - mean (190ms)  : 185, 195
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (7750) - mean (1,156ms)  : 1077, 1234
     .   : milestone, 1156,
    master - mean (1,148ms)  : 1083, 1212
     .   : milestone, 1148,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7750) - mean (272ms)  : 264, 280
     .   : milestone, 272,
    master - mean (272ms)  : 263, 280
     .   : milestone, 272,

    section Baseline
    This PR (7750) - mean (272ms)  : 264, 279
     .   : milestone, 272,
    master - mean (271ms)  : 263, 280
     .   : milestone, 271,

    section CallTarget+Inlining+NGEN
    This PR (7750) - mean (934ms)  : 885, 982
     .   : milestone, 934,
    master - mean (934ms)  : 895, 974
     .   : milestone, 934,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7750) - mean (266ms)  : 261, 271
     .   : milestone, 266,
    master - mean (263ms)  : 257, 269
     .   : milestone, 263,

    section Baseline
    This PR (7750) - mean (266ms)  : 260, 272
     .   : milestone, 266,
    master - mean (265ms)  : 257, 272
     .   : milestone, 265,

    section CallTarget+Inlining+NGEN
    This PR (7750) - mean (925ms)  : 865, 986
     .   : milestone, 925,
    master - mean (919ms)  : 865, 973
     .   : milestone, 919,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7750) - mean (265ms)  : 260, 271
     .   : milestone, 265,
    master - mean (265ms)  : 260, 270
     .   : milestone, 265,

    section Baseline
    This PR (7750) - mean (267ms)  : 262, 271
     .   : milestone, 267,
    master - mean (263ms)  : 257, 269
     .   : milestone, 263,

    section CallTarget+Inlining+NGEN
    This PR (7750) - mean (844ms)  : 824, 864
     .   : milestone, 844,
    master - mean (842ms)  : 820, 864
     .   : milestone, 842,
Loading

@datadog-official
Copy link

datadog-official bot commented Nov 3, 2025 •
edited by datadog-datadog-prod-us1 bot
Loading

⚠️ Tests

⚠️ Warnings

❄️ 21 New flaky tests detected

IntegrationDisabled from Datadog.Trace.ClrProfiler.IntegrationTests.AdoNet.MicrosoftDataSqlClientTests (Datadog) 
Expected exit code: 0, actual exit code: 134.
SubmitsTraces from Datadog.Trace.ClrProfiler.IntegrationTests.AdoNet.MicrosoftDataSqlClientTests (Datadog) 
Expected exit code: 0, actual exit code: 134.
SubmitsTraces from Datadog.Trace.ClrProfiler.IntegrationTests.AdoNet.MicrosoftDataSqlClientTests (Datadog) 
Expected exit code: 0, actual exit code: 134.
View all

ℹ️ Info

🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f13de89 | Docs | Datadog PR Page | Was this helpful? Give us feedback!

@pr-commenter
Copy link

pr-commenter bot commented Nov 3, 2025

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7750 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.253
  • 9 benchmarks have fewer allocations
  • 8 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7750

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.1 KB 6.05 KB -41 B -0.67%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 10.6μs 56.7ns 295ns 0 0 0 5.52 KB
master StartStopWithChild netcoreapp3.1 13.5μs 71.5ns 365ns 0 0 0 5.74 KB
master StartStopWithChild net472 22.3μs 122ns 755ns 0.904 0.201 0 6.1 KB
#7750 StartStopWithChild net6.0 10.9μs 55.6ns 249ns 0 0 0 5.51 KB
#7750 StartStopWithChild netcoreapp3.1 14μs 71.7ns 359ns 0 0 0 5.72 KB
#7750 StartStopWithChild net472 21.4μs 121ns 841ns 0.975 0.325 0.108 6.05 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 943μs 196ns 758ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.05ms 141ns 528ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.21ms 46.9ns 175ns 0 0 0 3.31 KB
#7750 WriteAndFlushEnrichedTraces net6.0 937μs 80ns 299ns 0 0 0 2.7 KB
#7750 WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 336ns 1.3μs 0 0 0 2.7 KB
#7750 WriteAndFlushEnrichedTraces net472 1.2ms 925ns 3.58μs 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Unknown 🤷 More allocations ⚠️

More allocations ⚠️ in #7750

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net6.0 0 b 1.22 KB 1.22 KB ∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑netcoreapp3.1 0 b 1.2 KB 1.2 KB ∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 0 b 1.23 KB 1.23 KB ∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net6.0 0 b 4.72 KB 4.72 KB ∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑netcoreapp3.1 0 b 4.62 KB 4.62 KB ∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472 0 b 4.74 KB 4.74 KB ∞

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleSimpleBody netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleSimpleBody net472 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleMoreComplexBody net6.0 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleMoreComplexBody netcoreapp3.1 N/A N/A N/A NaN NaN NaN 0 b
master AllCycleMoreComplexBody net472 N/A N/A N/A NaN NaN NaN 0 b
master ObjectExtractorSimpleBody net6.0 319ns 0.234ns 0.908ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 398ns 2.1ns 10.1ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 299ns 0.0358ns 0.129ns 0.0437 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.32μs 30.5ns 129ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.96μs 5.99ns 22.4ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.7μs 4.93ns 19.1ns 0.6 0 0 3.8 KB
#7750 AllCycleSimpleBody net6.0 1.07μs 5.46ns 24.4ns 0 0 0 1.22 KB
#7750 AllCycleSimpleBody netcoreapp3.1 1.4μs 7.65ns 45.3ns 0 0 0 1.2 KB
#7750 AllCycleSimpleBody net472 1.03μs 0.408ns 1.52ns 0.193 0 0 1.23 KB
#7750 AllCycleMoreComplexBody net6.0 7.06μs 32.7ns 131ns 0 0 0 4.72 KB
#7750 AllCycleMoreComplexBody netcoreapp3.1 8.85μs 44.2ns 198ns 0 0 0 4.62 KB
#7750 AllCycleMoreComplexBody net472 7.71μs 6.17ns 23.9ns 0.731 0 0 4.74 KB
#7750 ObjectExtractorSimpleBody net6.0 310ns 1.7ns 10.2ns 0 0 0 280 B
#7750 ObjectExtractorSimpleBody netcoreapp3.1 397ns 2.2ns 13.2ns 0 0 0 272 B
#7750 ObjectExtractorSimpleBody net472 295ns 0.0428ns 0.166ns 0.0445 0 0 281 B
#7750 ObjectExtractorMoreComplexBody net6.0 6.24μs 34.1ns 190ns 0 0 0 3.78 KB
#7750 ObjectExtractorMoreComplexBody netcoreapp3.1 7.78μs 37.3ns 154ns 0 0 0 3.69 KB
#7750 ObjectExtractorMoreComplexBody net472 6.7μs 8.67ns 33.6ns 0.57 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 76.5μs 21.2ns 79.3ns 0 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 98.2μs 80.6ns 302ns 0 0 0 32.4 KB
master EncodeArgs net472 113μs 13.6ns 52.6ns 5.07 0 0 32.51 KB
master EncodeLegacyArgs net6.0 143μs 20.6ns 74.2ns 0 0 0 2.15 KB
master EncodeLegacyArgs netcoreapp3.1 202μs 389ns 1.51μs 0 0 0 2.14 KB
master EncodeLegacyArgs net472 262μs 24.8ns 92.7ns 0 0 0 2.17 KB
#7750 EncodeArgs net6.0 76.8μs 164ns 612ns 0 0 0 32.4 KB
#7750 EncodeArgs netcoreapp3.1 97.5μs 16.3ns 62.9ns 0 0 0 32.4 KB
#7750 EncodeArgs net472 110μs 20.1ns 75.3ns 4.94 0 0 32.51 KB
#7750 EncodeLegacyArgs net6.0 145μs 29.1ns 109ns 0 0 0 2.15 KB
#7750 EncodeLegacyArgs netcoreapp3.1 198μs 187ns 725ns 0 0 0 2.14 KB
#7750 EncodeLegacyArgs net472 266μs 58.8ns 228ns 0 0 0 2.16 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 400μs 36.6ns 142ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 771μs 14.4μs 144μs 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 429μs 78.2ns 303ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 285μs 31.1ns 116ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 689μs 11.7μs 116μs 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 308μs 35.6ns 138ns 0 0 0 2.29 KB
#7750 RunWafRealisticBenchmark net6.0 393μs 102ns 396ns 0 0 0 4.55 KB
#7750 RunWafRealisticBenchmark netcoreapp3.1 813μs 13.9μs 138μs 0 0 0 4.48 KB
#7750 RunWafRealisticBenchmark net472 427μs 33.6ns 130ns 0 0 0 4.66 KB
#7750 RunWafRealisticBenchmarkWithAttack net6.0 282μs 29.9ns 116ns 0 0 0 2.24 KB
#7750 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 657μs 12.3μs 123μs 0 0 0 2.22 KB
#7750 RunWafRealisticBenchmarkWithAttack net472 310μs 18.5ns 66.5ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61μs 52.2ns 202ns 0 0 0 14.52 KB
master SendRequest netcoreapp3.1 72.3μs 133ns 499ns 0 0 0 17.42 KB
master SendRequest net472 0.00696ns 0.00256ns 0.00992ns 0 0 0 0 b
#7750 SendRequest net6.0 60.9μs 120ns 450ns 0 0 0 14.52 KB
#7750 SendRequest netcoreapp3.1 73.8μs 407ns 2.37μs 0 0 0 17.42 KB
#7750 SendRequest net472 0.00641ns 0.00202ns 0.00783ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7750

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0 2 B 4 B 2 B 100.00%

Fewer allocations 🎉 in #7750

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net6.0 4 B 2 B -2 B -50.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net472 73 B 0 b -73 B -100.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net472 47 B 0 b -47 B -100.00%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master OriginalCharSlice net6.0 1.91ms 8.66μs 32.4μs 0 0 0 640.01 KB
master OriginalCharSlice netcoreapp3.1 2.15ms 7.97μs 30.9μs 0 0 0 640 KB
master OriginalCharSlice net472 2.66ms 272ns 941ns 100 0 0 641.95 KB
master OptimizedCharSlice net6.0 1.33ms 361ns 1.4μs 0 0 0 4 B
master OptimizedCharSlice netcoreapp3.1 1.82ms 395ns 1.53μs 0 0 0 1 B
master OptimizedCharSlice net472 1.94ms 278ns 1.08μs 0 0 0 73 B
master OptimizedCharSliceWithPool net6.0 799μs 31.2ns 121ns 0 0 0 2 B
master OptimizedCharSliceWithPool netcoreapp3.1 812μs 221ns 854ns 0 0 0 1 B
master OptimizedCharSliceWithPool net472 1.13ms 119ns 462ns 0 0 0 47 B
#7750 OriginalCharSlice net6.0 1.92ms 272ns 1.05μs 0 0 0 640.01 KB
#7750 OriginalCharSlice netcoreapp3.1 2.09ms 5.74μs 22.2μs 0 0 0 640 KB
#7750 OriginalCharSlice net472 2.61ms 193ns 723ns 100 0 0 641.95 KB
#7750 OptimizedCharSlice net6.0 1.35ms 476ns 1.84μs 0 0 0 2 B
#7750 OptimizedCharSlice netcoreapp3.1 1.65ms 306ns 1.19μs 0 0 0 1 B
#7750 OptimizedCharSlice net472 2.02ms 417ns 1.62μs 0 0 0 0 b
#7750 OptimizedCharSliceWithPool net6.0 833μs 19.9ns 74.6ns 0 0 0 4 B
#7750 OptimizedCharSliceWithPool netcoreapp3.1 809μs 49.9ns 193ns 0 0 0 1 B
#7750 OptimizedCharSliceWithPool net472 1.13ms 121ns 470ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Faster 🎉 Fewer allocations 🎉

Faster 🎉 in #7750

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 1.361 893,067.93 656,348.66 several?

Fewer allocations 🎉 in #7750

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 42.76 KB 42.22 KB -542 B -1.27%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 707μs 2.59μs 10μs 0 0 0 41.74 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 871μs 5.02μs 41.1μs 0 0 0 42.76 KB
master WriteAndFlushEnrichedTraces net472 966μs 3.44μs 13.3μs 4.46 0 0 56 KB
#7750 WriteAndFlushEnrichedTraces net6.0 762μs 594ns 2.06μs 0 0 0 41.77 KB
#7750 WriteAndFlushEnrichedTraces netcoreapp3.1 661μs 3.59μs 20.3μs 0 0 0 42.22 KB
#7750 WriteAndFlushEnrichedTraces net472 895μs 4.29μs 18.2μs 8.33 0 0 55.95 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.91μs 2.23ns 8.04ns 0 0 0 1.02 KB
master ExecuteNonQuery netcoreapp3.1 2.67μs 12.7ns 53.7ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.88μs 3.56ns 13.8ns 0.144 0.0144 0 987 B
#7750 ExecuteNonQuery net6.0 1.88μs 7.66ns 29.7ns 0 0 0 1.02 KB
#7750 ExecuteNonQuery netcoreapp3.1 2.69μs 4.54ns 17.6ns 0 0 0 1.02 KB
#7750 ExecuteNonQuery net472 2.92μs 4.8ns 18.6ns 0.145 0.0145 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.73μs 7.36ns 28.5ns 0 0 0 1.03 KB
master CallElasticsearch netcoreapp3.1 2.21μs 8.7ns 32.6ns 0 0 0 1.03 KB
master CallElasticsearch net472 3.6μs 2.24ns 8.38ns 0.163 0 0 1.04 KB
master CallElasticsearchAsync net6.0 1.89μs 1.89ns 7.34ns 0 0 0 1.01 KB
master CallElasticsearchAsync netcoreapp3.1 2.31μs 10.3ns 39.8ns 0 0 0 1.08 KB
master CallElasticsearchAsync net472 3.76μs 2.11ns 8.19ns 0.169 0 0 1.1 KB
#7750 CallElasticsearch net6.0 1.7μs 8.57ns 41.1ns 0 0 0 1.03 KB
#7750 CallElasticsearch netcoreapp3.1 2.18μs 8.95ns 34.7ns 0 0 0 1.03 KB
#7750 CallElasticsearch net472 3.57μs 0.769ns 2.98ns 0.161 0 0 1.04 KB
#7750 CallElasticsearchAsync net6.0 1.82μs 2.05ns 7.69ns 0 0 0 1.01 KB
#7750 CallElasticsearchAsync netcoreapp3.1 2.3μs 10.6ns 42.5ns 0 0 0 1.08 KB
#7750 CallElasticsearchAsync net472 3.62μs 2.13ns 7.96ns 0.163 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.9μs 3.64ns 14.1ns 0 0 0 952 B
master ExecuteAsync netcoreapp3.1 2.46μs 3.83ns 14.8ns 0 0 0 952 B
master ExecuteAsync net472 2.57μs 1.3ns 5.02ns 0.141 0 0 915 B
#7750 ExecuteAsync net6.0 1.85μs 2.87ns 11.1ns 0 0 0 952 B
#7750 ExecuteAsync netcoreapp3.1 2.52μs 4.61ns 17.9ns 0 0 0 952 B
#7750 ExecuteAsync net472 2.6μs 2.02ns 7.82ns 0.143 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.91μs 16.1ns 62.4ns 0 0 0 2.36 KB
master SendAsync netcoreapp3.1 8.84μs 11.8ns 45.9ns 0 0 0 2.9 KB
master SendAsync net472 12.4μs 12.2ns 47.4ns 0.495 0 0 3.18 KB
#7750 SendAsync net6.0 6.99μs 19ns 71.1ns 0 0 0 2.36 KB
#7750 SendAsync netcoreapp3.1 8.65μs 23.3ns 90.1ns 0 0 0 2.9 KB
#7750 SendAsync net472 12.3μs 4.47ns 16.1ns 0.49 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Faster 🎉 More allocations ⚠️

Faster 🎉 in #7750

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 1.154 520,950.00 451,300.00

More allocations ⚠️ in #7750

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 278.53 KB 280.98 KB 2.45 KB 0.88%

Fewer allocations 🎉 in #7750

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 44.11 KB 43.44 KB -672 B -1.52%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 274 KB 255.4 KB -18.6 KB -6.79%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 279.14 KB 257.34 KB -21.8 KB -7.81%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 65.54 KB 57.34 KB -8.19 KB -12.50%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 47.9μs 328ns 3.11μs 0 0 0 44.11 KB
master StringConcatBenchmark netcoreapp3.1 51.6μs 304ns 2.73μs 0 0 0 42.87 KB
master StringConcatBenchmark net472 58.2μs 158ns 589ns 0 0 0 65.54 KB
master StringConcatAspectBenchmark net6.0 523μs 1.75μs 6.31μs 0 0 0 279.14 KB
master StringConcatAspectBenchmark netcoreapp3.1 530μs 1.04μs 3.76μs 0 0 0 274 KB
master StringConcatAspectBenchmark net472 396μs 1.34μs 4.84μs 0 0 0 278.53 KB
#7750 StringConcatBenchmark net6.0 45.9μs 622ns 6.13μs 0 0 0 43.44 KB
#7750 StringConcatBenchmark netcoreapp3.1 50.4μs 430ns 4.08μs 0 0 0 42.89 KB
#7750 StringConcatBenchmark net472 57.7μs 181ns 677ns 0 0 0 57.34 KB
#7750 StringConcatAspectBenchmark net6.0 454μs 2.09μs 8.37μs 0 0 0 257.34 KB
#7750 StringConcatAspectBenchmark netcoreapp3.1 498μs 851ns 3.07μs 0 0 0 255.4 KB
#7750 StringConcatAspectBenchmark net472 402μs 2.31μs 18μs 0 0 0 280.98 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.71μs 10.9ns 42.2ns 0 0 0 1.7 KB
master EnrichedLog netcoreapp3.1 3.6μs 16.1ns 62.3ns 0 0 0 1.7 KB
master EnrichedLog net472 4.02μs 7.35ns 28.5ns 0.244 0 0 1.64 KB
#7750 EnrichedLog net6.0 2.7μs 14.3ns 75.6ns 0 0 0 1.7 KB
#7750 EnrichedLog netcoreapp3.1 3.54μs 16ns 64.1ns 0 0 0 1.7 KB
#7750 EnrichedLog net472 4.11μs 8.99ns 34.8ns 0.247 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 125μs 44.1ns 159ns 0 0 0 4.31 KB
master EnrichedLog netcoreapp3.1 130μs 233ns 841ns 0 0 0 4.31 KB
master EnrichedLog net472 183μs 50ns 187ns 0 0 0 4.52 KB
#7750 EnrichedLog net6.0 123μs 120ns 433ns 0 0 0 4.31 KB
#7750 EnrichedLog netcoreapp3.1 128μs 197ns 765ns 0 0 0 4.31 KB
#7750 EnrichedLog net472 167μs 179ns 692ns 0 0 0 4.52 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.95μs 5.57ns 21.6ns 0 0 0 2.26 KB
master EnrichedLog netcoreapp3.1 6.64μs 13.5ns 52.4ns 0 0 0 2.26 KB
master EnrichedLog net472 7.58μs 4.07ns 15.8ns 0.303 0 0 2.08 KB
#7750 EnrichedLog net6.0 5.11μs 19.7ns 70.9ns 0 0 0 2.26 KB
#7750 EnrichedLog netcoreapp3.1 6.59μs 28.8ns 112ns 0 0 0 2.26 KB
#7750 EnrichedLog net472 7.71μs 8.37ns 32.4ns 0.306 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.07μs 9.96ns 39.8ns 0 0 0 1.2 KB
master SendReceive netcoreapp3.1 2.69μs 9.77ns 37.8ns 0 0 0 1.2 KB
master SendReceive net472 3.06μs 4.61ns 17.9ns 0.182 0 0 1.2 KB
#7750 SendReceive net6.0 2.07μs 10.7ns 47.9ns 0 0 0 1.2 KB
#7750 SendReceive netcoreapp3.1 2.64μs 13.7ns 62.8ns 0 0 0 1.2 KB
#7750 SendReceive net472 3.19μs 5.42ns 21ns 0.19 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.28μs 15.3ns 57.1ns 0 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 5.6μs 12.5ns 48.4ns 0 0 0 1.63 KB
master EnrichedLog net472 6.55μs 7.42ns 26.8ns 0.294 0 0 2.03 KB
#7750 EnrichedLog net6.0 4.24μs 16ns 62.2ns 0 0 0 1.58 KB
#7750 EnrichedLog netcoreapp3.1 5.57μs 11.7ns 45.5ns 0 0 0 1.63 KB
#7750 EnrichedLog net472 6.62μs 9.78ns 37.9ns 0.296 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 778ns 0.161ns 0.559ns 0 0 0 576 B
master StartFinishSpan netcoreapp3.1 961ns 4.84ns 24.7ns 0 0 0 576 B
master StartFinishSpan net472 919ns 0.516ns 2ns 0.089 0 0 578 B
master StartFinishScope net6.0 948ns 4.34ns 16.8ns 0 0 0 696 B
master StartFinishScope netcoreapp3.1 1.19μs 6.09ns 23.6ns 0 0 0 696 B
master StartFinishScope net472 1.13μs 1.82ns 7.05ns 0.103 0 0 658 B
#7750 StartFinishSpan net6.0 786ns 0.415ns 1.55ns 0 0 0 576 B
#7750 StartFinishSpan netcoreapp3.1 981ns 4.5ns 18ns 0 0 0 576 B
#7750 StartFinishSpan net472 930ns 1.85ns 7.15ns 0.0883 0 0 578 B
#7750 StartFinishScope net6.0 924ns 5.17ns 31.4ns 0 0 0 696 B
#7750 StartFinishScope netcoreapp3.1 1.17μs 6.1ns 30.5ns 0 0 0 696 B
#7750 StartFinishScope net472 1.12μs 0.779ns 3.02ns 0.101 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.06μs 0.535ns 2.07ns 0 0 0 696 B
master RunOnMethodBegin netcoreapp3.1 1.48μs 5.49ns 21.2ns 0 0 0 696 B
master RunOnMethodBegin net472 1.45μs 1.2ns 4.15ns 0.101 0 0 658 B
#7750 RunOnMethodBegin net6.0 1.04μs 0.905ns 3.39ns 0 0 0 696 B
#7750 RunOnMethodBegin netcoreapp3.1 1.46μs 6.81ns 26.4ns 0 0 0 696 B
#7750 RunOnMethodBegin net472 1.43μs 0.671ns 2.51ns 0.101 0 0 658 B

NachoEchevarria
NachoEchevarria approved these changes Nov 3, 2025
View reviewed changes
Copy link
Collaborator

@NachoEchevarria NachoEchevarria left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@andrewlock andrewlock merged commit eb9acab into master Nov 3, 2025
153 checks passed
@andrewlock andrewlock deleted the andrew/fix-asm-benchmarks branch November 3, 2025 12:59
@github-actions github-actions bot added this to the vNext-v3 milestone Nov 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chatgpt-codex-connector chatgpt-codex-connector[bot] chatgpt-codex-connector[bot] left review comments

@NachoEchevarria NachoEchevarria NachoEchevarria approved these changes

Assignees

No one assigned

Labels

area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc area:tests unit tests, integration tests type:flake-fix

Projects

None yet

Milestone

3.31.0

Development

Successfully merging this pull request may close these issues.

3 participants

@andrewlock @NachoEchevarria