[NO-TICKET] Workaround profiling benchmark flakiness on Ruby 2.6 #5077

ivoanjo · 2025-11-21T14:44:43Z

What does this PR do?

This PR skips the profiling/validate_benchmarks_spec.rb on Ruby 2.6 as these have proven flaky and we've not been able to figure out why yet.

The objective of this spec is to validate that the benchmarks don't bitrot -- nothing else, so for now let's skip running them on Ruby 2.6 and rely on the fact they do run on other Ruby versions to keep them working (and maybe someday get to the bottom of this issue).

This removes this issue as a common source of CI flakiness.

Motivation:

For a while we've been hunting some flakiness in the profiling benchmarks that seems to especially affect Ruby 2.6 -- see #4947 .

The mechanism we added on that PR didn't particularly help diagnose what's going on:

  1) Profiling benchmarks profiling_sample_loop_v2 runs without raising errors
     Failure/Error: raise "Failure or timeout in `expect_in_fork`#{crash_note}, STDOUT: `#{stdout}`, STDERR: `#{stderr}`", cause: e

     RuntimeError:
       Failure or timeout in `expect_in_fork` (Crashing Ruby to get stacktrace as requested by `trigger_stacktrace_on_kill`), STDOUT: `Current pid is 4727
       Calculating -------------------------------------
       profiling - stack collector (ruby frames - native filenames enabled)
                                 4.240k (±13.4%) i/s -     39.000  in   0.010075s
       `, STDERR: `/__w/dd-trace-rb/dd-trace-rb/benchmarks/profiling_sample_loop_v2.rb:16: [BUG] Segmentation fault at 0x000000000000124f
       ruby 2.6.10p210 (2022-04-12 revision 67958) [x86_64-linux]

       -- Control frame information -----------------------------------------------
       c:0004 p:---- s:0014 e:000013 CFUNC  :sleep
       c:0003 p:0005 s:0010 e:000009 BLOCK  /__w/dd-trace-rb/dd-trace-rb/benchmarks/profiling_sample_loop_v2.rb:16
       c:0002 p:0011 s:0007 e:000006 BLOCK  /__w/dd-trace-rb/dd-trace-rb/spec/spec_helper.rb:290 [FINISH]
       c:0001 p:---- s:0003 e:000002 (none) [FINISH]

       -- Ruby level backtrace information ----------------------------------------
       /__w/dd-trace-rb/dd-trace-rb/spec/spec_helper.rb:290:in `block in initialize'
       /__w/dd-trace-rb/dd-trace-rb/benchmarks/profiling_sample_loop_v2.rb:16:in `block in <class:ProfilerSampleLoopBenchmark>'
       /__w/dd-trace-rb/dd-trace-rb/benchmarks/profiling_sample_loop_v2.rb:16:in `sleep'

       -- Machine register context ------------------------------------------------
        RIP: 0x00007f7d3169bb16 RBP: 0x00005613aa1949e0 RSP: 0x00007ffc8b1f59a0
        RAX: 0xfffffffffffffffc RBX: 0x00005613aa194a08 RCX: 0x00007f7d3169bb16
        RDX: 0x0000000000000000 RDI: 0x00007ffc8b1f5a40 RSI: 0x0000000000000002
         R8: 0x0000000000000008  R9: 0x0000000000000000 R10: 0x0000000000000000
        R11: 0x0000000000000293 R12: 0x0000000000000000 R13: 0x00005613b05201f0
        R14: 0x0000000000000000 R15: 0x00007f7d3106fec8 EFL: 0x0000000000000293

       -- C level backtrace information -------------------------------------------
       /usr/local/lib/libruby.so.2.6(rb_print_backtrace+0x11) [0x7f7d319f742a] vm_dump.c:715
       /usr/local/lib/libruby.so.2.6(rb_vm_bugreport) vm_dump.c:985
       /usr/local/lib/libruby.so.2.6(rb_bug_context+0xec) [0x7f7d31847cec] error.c:609
       /usr/local/lib/libruby.so.2.6(sigsegv+0x42) [0x7f7d3195e752] signal.c:998
       /usr/local/bundle/gems/libdatadog-24.0.1.1.0-x86_64-linux/vendor/libdatadog-24.0.1/x86_64-linux/libdatadog-x86_64-unknown-linux-gnu/lib/pkgconfig/../../lib/libdatadog_profiling.so(_ZN18libdd_crashtracker9collector13crash_handler22handle_posix_sigaction17hccec31010ba0e887E+0x59e) [0x7f7d2dfc2110]
       /lib/x86_64-linux-gnu/libc.so.6(0x7f7d315e4de0) [0x7f7d315e4de0]
       /lib/x86_64-linux-gnu/libc.so.6(ppoll+0xa6) [0x7f7d3169bb16]
       /usr/local/lib/libruby.so.2.6(native_ppoll_sleep+0x19a) [0x7f7d319a059a] thread_pthread.c:2131
       /usr/local/lib/libruby.so.2.6(native_sleep+0x2fa) [0x7f7d319a10ea] thread_pthread.c:2170
       /usr/local/lib/libruby.so.2.6(thread_join_sleep+0x12f) [0x7f7d319a454f] thread.c:987
       /usr/local/lib/libruby.so.2.6(rb_ensure+0xce) [0x7f7d3184fa3e] eval.c:1088
       /usr/local/lib/libruby.so.2.6(thread_join+0x38) [0x7f7d3199d178] thread.c:1032
       /usr/local/lib/libruby.so.2.6(thread_join_m) thread.c:1129
       /usr/local/lib/libruby.so.2.6(vm_call0_cfunc_with_frame+0x85) [0x7f7d319e85cf] vm_eval.c:86
       /usr/local/lib/libruby.so.2.6(vm_call0_cfunc) vm_eval.c:100
       /usr/local/lib/libruby.so.2.6(vm_call0_body) vm_eval.c:131
       /usr/local/lib/libruby.so.2.6(rb_vm_call0) vm_eval.c:60
       /usr/local/lib/libruby.so.2.6(invoke_block_from_c_bh+0x1f) [0x7f7d319f18ef] vm_insnhelper.c:2555
       /usr/local/lib/libruby.so.2.6(vm_yield) vm.c:1155
       /usr/local/lib/libruby.so.2.6(rb_yield_0) vm_eval.c:978
       /usr/local/lib/libruby.so.2.6(rb_yield_1) vm_eval.c:984
       /usr/local/lib/libruby.so.2.6(rb_yield) vm_eval.c:994
       /usr/local/lib/libruby.so.2.6(rb_array_len+0x0) [0x7f7d317be1ac] array.c:2087
       /usr/local/lib/libruby.so.2.6(rb_ary_each) array.c:2086
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(rb_load_internal0+0xff) [0x7f7d3189be8f] load.c:612
       /usr/local/lib/libruby.so.2.6(rb_load_internal+0xb) [0x7f7d3189c293] load.c:643
       /usr/local/lib/libruby.so.2.6(rb_f_load) load.c:711
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x148) [0x7f7d319dea38] insns.def:765
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(rb_protect+0x1e5) [0x7f7d3184f7f5] eval.c:1046
       /usr/local/lib/libruby.so.2.6(rb_f_fork+0x16) [0x7f7d3191502f] process.c:4139
       /usr/local/lib/libruby.so.2.6(rb_f_fork) process.c:4130
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x1e05) [0x7f7d319e06f5] insns.def:836
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x856) [0x7f7d319e4d66] vm.c:1894
       /usr/local/lib/libruby.so.2.6(rb_yield+0x276) [0x7f7d319f15e6] vm.c:1021
       /usr/local/lib/libruby.so.2.6(rb_ary_collect+0x5c) [0x7f7d317c3dac] array.c:3017
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(rb_yield+0x276) [0x7f7d319f15e6] vm.c:1021
       /usr/local/lib/libruby.so.2.6(rb_ary_collect+0x5c) [0x7f7d317c3dac] array.c:3017
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(rb_yield+0x276) [0x7f7d319f15e6] vm.c:1021
       /usr/local/lib/libruby.so.2.6(rb_ary_collect+0x5c) [0x7f7d317c3dac] array.c:3017
       /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910
       /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924
       /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750
       /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885
       /usr/local/lib/libruby.so.2.6(ruby_exec_internal+0xc4) [0x7f7d3184c4a4] eval.c:262
       /usr/local/lib/libruby.so.2.6(ruby_exec_node+0x1d) [0x7f7d3184e96d] eval.c:326
       /usr/local/lib/libruby.so.2.6(ruby_run_node+0x2e) [0x7f7d3185161e] eval.c:318
       /usr/local/bin/ruby(main+0x5b) [0x56137eb1310b] ./main.c:42

Change log entry

None.

Additional Notes:

N/A

How to test the change?

Validate spec still runs on all Rubies other than 2.6.

**What does this PR do?** This PR skips the `profiling/validate_benchmarks_spec.rb` on Ruby 2.6 as these have proven flaky and we've not been able to figure out why yet. The objective of this spec is to validate that the benchmarks don't bitrot -- nothing else, so for now let's skip running them on Ruby 2.6 and rely on the fact they do run on other Ruby versions to keep them working (and maybe someday get to the bottom of this issue). This removes this issue as a common source of CI flakiness. **Motivation:** For a while we've been hunting some flakiness in the profiling benchmarks that seems to especially affect Ruby 2.6 -- see #4947 . The mechanism we added on that PR didn't particularly help diagnose what's going on: ``` 1) Profiling benchmarks profiling_sample_loop_v2 runs without raising errors Failure/Error: raise "Failure or timeout in `expect_in_fork`#{crash_note}, STDOUT: `#{stdout}`, STDERR: `#{stderr}`", cause: e RuntimeError: Failure or timeout in `expect_in_fork` (Crashing Ruby to get stacktrace as requested by `trigger_stacktrace_on_kill`), STDOUT: `Current pid is 4727 Calculating ------------------------------------- profiling - stack collector (ruby frames - native filenames enabled) 4.240k (±13.4%) i/s - 39.000 in 0.010075s `, STDERR: `/__w/dd-trace-rb/dd-trace-rb/benchmarks/profiling_sample_loop_v2.rb:16: [BUG] Segmentation fault at 0x000000000000124f ruby 2.6.10p210 (2022-04-12 revision 67958) [x86_64-linux] -- Control frame information ----------------------------------------------- c:0004 p:---- s:0014 e:000013 CFUNC :sleep c:0003 p:0005 s:0010 e:000009 BLOCK /__w/dd-trace-rb/dd-trace-rb/benchmarks/profiling_sample_loop_v2.rb:16 c:0002 p:0011 s:0007 e:000006 BLOCK /__w/dd-trace-rb/dd-trace-rb/spec/spec_helper.rb:290 [FINISH] c:0001 p:---- s:0003 e:000002 (none) [FINISH] -- Ruby level backtrace information ---------------------------------------- /__w/dd-trace-rb/dd-trace-rb/spec/spec_helper.rb:290:in `block in initialize' /__w/dd-trace-rb/dd-trace-rb/benchmarks/profiling_sample_loop_v2.rb:16:in `block in <class:ProfilerSampleLoopBenchmark>' /__w/dd-trace-rb/dd-trace-rb/benchmarks/profiling_sample_loop_v2.rb:16:in `sleep' -- Machine register context ------------------------------------------------ RIP: 0x00007f7d3169bb16 RBP: 0x00005613aa1949e0 RSP: 0x00007ffc8b1f59a0 RAX: 0xfffffffffffffffc RBX: 0x00005613aa194a08 RCX: 0x00007f7d3169bb16 RDX: 0x0000000000000000 RDI: 0x00007ffc8b1f5a40 RSI: 0x0000000000000002 R8: 0x0000000000000008 R9: 0x0000000000000000 R10: 0x0000000000000000 R11: 0x0000000000000293 R12: 0x0000000000000000 R13: 0x00005613b05201f0 R14: 0x0000000000000000 R15: 0x00007f7d3106fec8 EFL: 0x0000000000000293 -- C level backtrace information ------------------------------------------- /usr/local/lib/libruby.so.2.6(rb_print_backtrace+0x11) [0x7f7d319f742a] vm_dump.c:715 /usr/local/lib/libruby.so.2.6(rb_vm_bugreport) vm_dump.c:985 /usr/local/lib/libruby.so.2.6(rb_bug_context+0xec) [0x7f7d31847cec] error.c:609 /usr/local/lib/libruby.so.2.6(sigsegv+0x42) [0x7f7d3195e752] signal.c:998 /usr/local/bundle/gems/libdatadog-24.0.1.1.0-x86_64-linux/vendor/libdatadog-24.0.1/x86_64-linux/libdatadog-x86_64-unknown-linux-gnu/lib/pkgconfig/../../lib/libdatadog_profiling.so(_ZN18libdd_crashtracker9collector13crash_handler22handle_posix_sigaction17hccec31010ba0e887E+0x59e) [0x7f7d2dfc2110] /lib/x86_64-linux-gnu/libc.so.6(0x7f7d315e4de0) [0x7f7d315e4de0] /lib/x86_64-linux-gnu/libc.so.6(ppoll+0xa6) [0x7f7d3169bb16] /usr/local/lib/libruby.so.2.6(native_ppoll_sleep+0x19a) [0x7f7d319a059a] thread_pthread.c:2131 /usr/local/lib/libruby.so.2.6(native_sleep+0x2fa) [0x7f7d319a10ea] thread_pthread.c:2170 /usr/local/lib/libruby.so.2.6(thread_join_sleep+0x12f) [0x7f7d319a454f] thread.c:987 /usr/local/lib/libruby.so.2.6(rb_ensure+0xce) [0x7f7d3184fa3e] eval.c:1088 /usr/local/lib/libruby.so.2.6(thread_join+0x38) [0x7f7d3199d178] thread.c:1032 /usr/local/lib/libruby.so.2.6(thread_join_m) thread.c:1129 /usr/local/lib/libruby.so.2.6(vm_call0_cfunc_with_frame+0x85) [0x7f7d319e85cf] vm_eval.c:86 /usr/local/lib/libruby.so.2.6(vm_call0_cfunc) vm_eval.c:100 /usr/local/lib/libruby.so.2.6(vm_call0_body) vm_eval.c:131 /usr/local/lib/libruby.so.2.6(rb_vm_call0) vm_eval.c:60 /usr/local/lib/libruby.so.2.6(invoke_block_from_c_bh+0x1f) [0x7f7d319f18ef] vm_insnhelper.c:2555 /usr/local/lib/libruby.so.2.6(vm_yield) vm.c:1155 /usr/local/lib/libruby.so.2.6(rb_yield_0) vm_eval.c:978 /usr/local/lib/libruby.so.2.6(rb_yield_1) vm_eval.c:984 /usr/local/lib/libruby.so.2.6(rb_yield) vm_eval.c:994 /usr/local/lib/libruby.so.2.6(rb_array_len+0x0) [0x7f7d317be1ac] array.c:2087 /usr/local/lib/libruby.so.2.6(rb_ary_each) array.c:2086 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(rb_load_internal0+0xff) [0x7f7d3189be8f] load.c:612 /usr/local/lib/libruby.so.2.6(rb_load_internal+0xb) [0x7f7d3189c293] load.c:643 /usr/local/lib/libruby.so.2.6(rb_f_load) load.c:711 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x148) [0x7f7d319dea38] insns.def:765 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(rb_protect+0x1e5) [0x7f7d3184f7f5] eval.c:1046 /usr/local/lib/libruby.so.2.6(rb_f_fork+0x16) [0x7f7d3191502f] process.c:4139 /usr/local/lib/libruby.so.2.6(rb_f_fork) process.c:4130 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x1e05) [0x7f7d319e06f5] insns.def:836 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(yield_under+0x3e6) [0x7f7d319f2b56] vm.c:1021 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x856) [0x7f7d319e4d66] vm.c:1894 /usr/local/lib/libruby.so.2.6(rb_yield+0x276) [0x7f7d319f15e6] vm.c:1021 /usr/local/lib/libruby.so.2.6(rb_ary_collect+0x5c) [0x7f7d317c3dac] array.c:3017 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(rb_yield+0x276) [0x7f7d319f15e6] vm.c:1021 /usr/local/lib/libruby.so.2.6(rb_ary_collect+0x5c) [0x7f7d317c3dac] array.c:3017 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(rb_yield+0x276) [0x7f7d319f15e6] vm.c:1021 /usr/local/lib/libruby.so.2.6(rb_ary_collect+0x5c) [0x7f7d317c3dac] array.c:3017 /usr/local/lib/libruby.so.2.6(vm_cfp_consistent_p+0x0) [0x7f7d319d415b] vm_insnhelper.c:1908 /usr/local/lib/libruby.so.2.6(vm_call_cfunc_with_frame) vm_insnhelper.c:1910 /usr/local/lib/libruby.so.2.6(vm_call_cfunc) vm_insnhelper.c:1924 /usr/local/lib/libruby.so.2.6(vm_exec_core+0x234) [0x7f7d319deb24] insns.def:750 /usr/local/lib/libruby.so.2.6(rb_vm_exec+0x155) [0x7f7d319e4665] vm.c:1885 /usr/local/lib/libruby.so.2.6(ruby_exec_internal+0xc4) [0x7f7d3184c4a4] eval.c:262 /usr/local/lib/libruby.so.2.6(ruby_exec_node+0x1d) [0x7f7d3184e96d] eval.c:326 /usr/local/lib/libruby.so.2.6(ruby_run_node+0x2e) [0x7f7d3185161e] eval.c:318 /usr/local/bin/ruby(main+0x5b) [0x56137eb1310b] ./main.c:42 ``` **Additional Notes:** N/A **How to test the change?** Validate spec still runs on all Rubies other than 2.6.

datadog-datadog-prod-us1 · 2025-11-21T15:02:33Z

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage
• Patch Coverage: 100.00%
• Total Coverage: 98.52% (+0.05%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 8650e35 | Docs | Datadog PR Page | Was this helpful? Give us feedback!}

pr-commenter · 2025-11-21T15:14:39Z

Benchmarks

Benchmark execution time: 2025-11-21 15:14:16

Comparing candidate commit 8650e35 in PR branch ivoanjo/profiling-benchmarks-flaky-workaround with baseline commit 2c84fc0 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 44 metrics, 2 unstable metrics.

ivoanjo requested review from a team as code owners November 21, 2025 14:44

github-actions bot added the dev/testing Involves testing processes (e.g. RSpec) label Nov 21, 2025

lloeki approved these changes Nov 24, 2025

View reviewed changes

ivoanjo merged commit efaecc3 into master Nov 24, 2025
276 checks passed

ivoanjo deleted the ivoanjo/profiling-benchmarks-flaky-workaround branch November 24, 2025 12:06

github-actions bot added this to the 2.23.0 milestone Nov 24, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[NO-TICKET] Workaround profiling benchmark flakiness on Ruby 2.6 #5077

[NO-TICKET] Workaround profiling benchmark flakiness on Ruby 2.6 #5077

Uh oh!

ivoanjo commented Nov 21, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented Nov 21, 2025

Uh oh!

pr-commenter bot commented Nov 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

[NO-TICKET] Workaround profiling benchmark flakiness on Ruby 2.6 #5077

[NO-TICKET] Workaround profiling benchmark flakiness on Ruby 2.6 #5077

Uh oh!

Conversation

ivoanjo commented Nov 21, 2025

Uh oh!

datadog-datadog-prod-us1 bot commented Nov 21, 2025

Uh oh!

pr-commenter bot commented Nov 21, 2025

Benchmarks

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants