Skip to content

Add old rubies to onboarding #5736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Add old rubies to onboarding #5736

wants to merge 2 commits into from

Conversation

@lloeki
Copy link
Member

@lloeki lloeki commented Nov 19, 2025 •
edited
Loading

Motivation

Add coverage for Ruby versions

Changes

Add Ruby 2.6

Chained behind #5735

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • If PR title starts with [<language>], double-check that only <language> is impacted by the change
  • No system-tests internal is modified. Otherwise, I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added (or removed)?

@lloeki lloeki changed the base branch from main to lloeki/add-new-rubies-to-onboarding November 19, 2025 08:50
@github-actions
Copy link
Contributor

github-actions bot commented Nov 19, 2025 •
edited
Loading

CODEOWNERS have been resolved as:

lib-injection/build/docker/ruby/lib_injection_rails61_app/.gitattributes  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/.gitignore    @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/.ruby-version  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/Gemfile       @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/Gemfile.lock  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/README.md     @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/Rakefile      @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/assets/config/manifest.js  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/assets/images/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/assets/stylesheets/application.css  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/controllers/application_controller.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/controllers/concerns/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/controllers/datadog_controller.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/helpers/application_helper.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/models/application_record.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/models/concerns/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/app/views/layouts/application.html.erb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/bin/bundle    @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/bin/rails     @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/bin/rake      @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/bin/setup     @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config.ru     @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/application.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/boot.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/credentials.yml.enc  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/database.yml  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/environment.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/environments/development.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/environments/production.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/environments/test.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/initializers/assets.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/initializers/content_security_policy.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/initializers/filter_parameter_logging.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/initializers/inflections.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/locales/en.yml  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/puma.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/config/routes.rb  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/db/seeds.rb   @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/lib/assets/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/lib/tasks/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/public/404.html  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/public/422.html  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/public/500.html  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/public/apple-touch-icon-precomposed.png  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/public/apple-touch-icon.png  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/public/favicon.ico  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/public/robots.txt  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/test/controllers/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/test/fixtures/files/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/test/helpers/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/test/integration/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/test/models/.keep  @DataDog/system-tests-core
lib-injection/build/docker/ruby/lib_injection_rails61_app/test/test_helper.rb  @DataDog/system-tests-core
utils/build/virtual_machine/weblogs/ruby/test-app-ruby-multialpine/Dockerfile.ruby_2_6-alpine  @DataDog/system-tests-core
utils/build/virtual_machine/weblogs/ruby/test-app-ruby-multicontainer/Dockerfile.ruby_2_6  @DataDog/system-tests-core
utils/build/virtual_machine/weblogs/ruby/provision_test-app-ruby-multialpine.yml  @DataDog/system-tests-core
utils/build/virtual_machine/weblogs/ruby/provision_test-app-ruby-multicontainer.yml  @DataDog/system-tests-core
utils/build/virtual_machine/weblogs/ruby/test-app-ruby-multialpine/docker-compose.yml  @DataDog/system-tests-core
utils/build/virtual_machine/weblogs/ruby/test-app-ruby-multialpine/nginx.conf  @DataDog/system-tests-core
utils/build/virtual_machine/weblogs/ruby/test-app-ruby-multicontainer/docker-compose.yml  @DataDog/system-tests-core
utils/build/virtual_machine/weblogs/ruby/test-app-ruby-multicontainer/nginx.conf  @DataDog/system-tests-core

@lloeki lloeki mentioned this pull request Nov 19, 2025
Merged
6 tasks
@lloeki lloeki force-pushed the lloeki/add-new-rubies-to-onboarding branch from 18d997a to 65d1a02 Compare November 19, 2025 10:55
@lloeki
Copy link
Member Author

lloeki commented Nov 19, 2025

Ruby 2.6 fails with:

    ------
     > [ruby_2_6 10/10] RUN bundle install:
    6.371 Fetching gem metadata from https://rubygems.org/..........
    14.10 Resolving dependencies...
    15.33 Resolving dependencies...
    26.13 Could not find compatible versions
    26.13
    26.13 Because rails >= 7.0.0.alpha1, < 7.2.0.beta1 depends on Ruby >= 2.7.0
    26.13   and Gemfile depends on rails ~> 7.0.3,
    26.13   Ruby >= 2.7.0 is required.
    26.13 So, because current Ruby version is = 2.6.10,
    26.13   version solving has failed.
    ------
    Dockerfile.ruby_2_6:41

    --------------------

      39 |

      40 |     # Install gems

      41 | >>> RUN bundle install

      42 |

      43 |     # Set entrypoint

    --------------------

    target ruby_2_6: failed to solve: process "/bin/sh -c bundle install" did not complete successfully: exit code: 6

  pulumi:pulumi:Stack (system-tests-vms-system-tests_onboarding):
    Downloading provider: aws
    Downloading provider: command
    error: update failed

Which is sort of expected because this app is Rails 7.0 and that needs Ruby 2.7.

@lloeki lloeki force-pushed the lloeki/add-old-rubies-to-onboarding branch 3 times, most recently from f8ccedd to 97f00a2 Compare November 19, 2025 13:24
@lloeki lloeki marked this pull request as ready for review November 19, 2025 13:24
@lloeki lloeki requested a review from a team as a code owner November 19, 2025 13:24
@lloeki lloeki force-pushed the lloeki/add-new-rubies-to-onboarding branch from 65d1a02 to 90a2059 Compare November 19, 2025 14:31
@lloeki lloeki requested review from a team as code owners November 19, 2025 14:31
@lloeki lloeki force-pushed the lloeki/add-old-rubies-to-onboarding branch from 97f00a2 to 2e89fd5 Compare November 19, 2025 14:34
@lloeki
Copy link
Member Author

lloeki commented Nov 19, 2025

Python issue?

Screenshot 2025-11-19 at 17 28 40

@lloeki
Copy link
Member Author

lloeki commented Nov 19, 2025 •
edited
Loading

K8S test flakiness? Or Python as above?
Screenshot 2025-11-19 at 17 27 39

EDIT: ah, there was some dotnet in there too.

@lloeki lloeki force-pushed the lloeki/add-new-rubies-to-onboarding branch from 90a2059 to 0fc3a3e Compare November 20, 2025 09:40
@lloeki lloeki force-pushed the lloeki/add-old-rubies-to-onboarding branch from 2e89fd5 to 274cbe9 Compare November 20, 2025 09:40
@lloeki lloeki force-pushed the lloeki/add-new-rubies-to-onboarding branch from 0fc3a3e to 2029dae Compare November 20, 2025 09:44
@lloeki lloeki force-pushed the lloeki/add-old-rubies-to-onboarding branch from 274cbe9 to b393c59 Compare November 20, 2025 09:48
Strech
Strech reviewed Nov 20, 2025
View reviewed changes
lib-injection/build/docker/ruby/lib_injection_rails61_app/Gemfile
gem "rails", "~> 6.1.7"

# The original asset pipeline for Rails [https://github.com/rails/sprockets-rails]
gem "sprockets-rails"
Copy link
Member

@Strech Strech Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can also ditch it :D

@lloeki lloeki force-pushed the lloeki/add-new-rubies-to-onboarding branch from 2029dae to 23ec1df Compare November 20, 2025 16:14
@lloeki lloeki force-pushed the lloeki/add-old-rubies-to-onboarding branch from b393c59 to 8f5b545 Compare November 20, 2025 16:16
@lloeki
Copy link
Member Author

lloeki commented Nov 21, 2025

It failed but I don't know what's happening.

There is no usable output on the failing job:

Screenshot 2025-11-21 at 11 43 28

I downloaded the artifacts, but the logs on the failing job are truncated:

Screenshot 2025-11-21 at 11 45 20

@lloeki
Copy link
Member Author

lloeki commented Nov 21, 2025 •
edited
Loading

I tried provisioning the VM locally but after some long time sitting there I have it crashing with a very strange error:

    error: python inline source runtime error: maximum recursion depth exceeded
    Traceback (most recent call last):
    Traceback (most recent call last):
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/automation/_server.py", line 80, in run
        loop.run_until_complete(run_in_stack(self.program))
      File "/nix/store/ybnf7k6i9p244bbhsbxizqk65z58cwyr-python3-3.12.6/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
        return future.result()
               ^^^^^^^^^^^^^^^
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/stack.py", line 138, in run_in_stack
        await run_pulumi_func(run)
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/stack.py", line 52, in run_pulumi_func
        await wait_for_rpcs()
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/stack.py", line 85, in wait_for_rpcs
        raise exn from cause
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/rpc_manager.py", line 71, in rpc_wrapper
        result = await rpc
                 ^^^^^^^^^
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/resource.py", line 910, in do_register
        resolver = await prepare_resource(res, ty, custom, remote, props, opts, typ)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/resource.py", line 181, in prepare_resource
        explicit_urn_dependencies = await _resolve_depends_on_urns(
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/resource.py", line 1210, in _resolve_depends_on_urns
        return await rpc._expand_dependencies(all_deps, from_resource)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/rpc.py", line 326, in _expand_dependencies
        await _add_dependency(urns, d, from_resource)
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/rpc.py", line 283, in _add_dependency
        no_cycles = declare_dependency(from_resource, res) if from_resource else True
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/resource_cycle_breaker.py", line 32, in declare_dependency
        if _reachable(from_resource=to_resource, to_resource=from_resource):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/resource_cycle_breaker.py", line 52, in _reachable
        for x in _with_transitive_deps(from_resource, visited):
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/resource_cycle_breaker.py", line 69, in _with_transitive_deps
        yield from _with_transitive_deps(x, visited)
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/resource_cycle_breaker.py", line 69, in _with_transitive_deps
        yield from _with_transitive_deps(x, visited)
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/resource_cycle_breaker.py", line 69, in _with_transitive_deps
        yield from _with_transitive_deps(x, visited)
      [Previous line repeated 976 more times]
      File "/Users/loic.nageleisen/Source/github.com/DataDog/system-tests/vendor/python/3.12/packages/lib/python3.12/site-packages/pulumi/runtime/resource_cycle_breaker.py", line 68, in _with_transitive_deps
        for x in _deps(r):
                 ^^^^^^^^
    RecursionError: maximum recursion depth exceeded
    error: Exception calling application: maximum recursion depth exceeded

@lloeki
Copy link
Member Author

lloeki commented Nov 21, 2025

Huh, I found this in logs_multi_installer_auto_injection/tests.log:

11:52:51.430 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (54s) Installing datadog-apm-inject...
11:52:51.666 INFO     @ updating....
11:52:51.666 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (54s) error: could not evaluate symlinks for stable link: lstat /opt/datadog-packages/datadog-apm-inject/stable: no such file or directory
11:52:52.245 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (55s) Successfully installed datadog-apm-inject
11:52:52.246 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (55s) Installing datadog-apm-library-ruby...
11:52:52.337 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (55s) {"error":"failed to install package oci://install.datadoghq.com/apm-library-ruby-package:latest: could not download package: could not download package: could not download image from any registry: could not download image using install.datadoghq.com/apm-library-ruby-package:latest: GET https://installtesting.datad0g.com/v2/apm-library-ruby-package/manifests/latest: unexpected status code 403 Forbidden: \u003c?xml version=\"1.0\" encoding=\"UTF-8\"?\u003e\n\u003cError\u003e\u003cCode\u003eAccessDenied\u003c/Code\u003e\u003cMessage\u003eAccess Denied\u003c/Message\u003e\u003c/Error\u003e","code":1}

----8<----

11:52:52.894 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (55s) curl: (22) The requested URL returned error: 403
11:52:52.901 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (55s) Unable to send telemetry
11:52:54.308 INFO     @ updating....
11:52:54.309 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (57s) curl: (22) The requested URL returned error: 403
11:52:54.312 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (57s) Unable to send logs telemetry\n
11:52:54.421 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (57s) curl: (22) The requested URL returned error: 403
11:52:54.421 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer creating (57s) Unable to send trace telemetry\n
11:52:57.173 INFO     @ updating......
11:52:57.175 INFO      +  command:remote:Command -Ubuntu_24_amd64-install-installer created (60s) Unable to send trace telemetry\n

----8<----

11:53:37.676 INFO     Diagnostics:
11:53:37.676 INFO       pulumi:pulumi:Stack (system-tests-vms-system-tests_onboarding):
11:53:37.676 INFO         warning: using pulumi-resource-aws from $PATH at /etc/profiles/per-user/loic.nageleisen/bin/pulumi-resource-aws
11:53:37.676 INFO         warning: using pulumi-resource-aws from $PATH at /etc/profiles/per-user/loic.nageleisen/bin/pulumi-resource-aws
11:53:37.677 INFO         error: python inline source runtime error: maximum recursion depth exceeded

Base automatically changed from lloeki/add-new-rubies-to-onboarding to main November 21, 2025 12:52
@lloeki lloeki force-pushed the lloeki/add-old-rubies-to-onboarding branch from 8f5b545 to 53f89d9 Compare November 21, 2025 14:05
@lloeki lloeki marked this pull request as draft November 21, 2025 17:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Strech Strech Strech approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@lloeki @Strech