Skip to content

Commit 54efa06

Browse files
TonyCTHsuy9vivoanjop-datadog
authored
Update document v2.12.0 (#4443)
* [🤖] Update Latest Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13477718964 * Remove force-executed tests and scenario from system-tests workflow after release 2.11.0 * Add AppSec RestClient instrumentation for SSRF detection * Add type signatures for AppSec RestClient integration * Fix type signatures for excon and faraday AppSec integrations * Add tests for RestClient AppSec instrumentation * Extract rest-client-latest appraisal * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13455470094 * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13455581146 * Remove unneeded variable from RestClient SSRF integration test * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13457056485 * Add rest-client gem to rake edge:update task * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13496444973 * Rename RequestPatch to RequestSSRFDetectionPatch * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13497053640 * Disable rubocop warning for RequestSSRFDetectionPatch * [PROF-11151] Fix test-memory-leaks flaky behavior **What does this PR do?** This PR will hopefully fix almost [all of the the flakiness](https://github.com/DataDog/dd-trace-rb/pulls?q=is%3Apr+asan) we've seen from the test-memory-leaks GitHub workflow. The previous "asan" builds we were using were built from ruby-head, which means that any instability or early breakage in ruby-head would make test-memory-leaks fail. To fix this, I've worked with upstream to create these 3.4-asan builds: these are Ruby builds that are built **from the latest 3.4 stable Ruby** with asan. Thus any breakages we see in them, should also exist in regular 3.4 builds. **Motivation:** With this change, the test-memory-leaks workflow becomes a lot more valuable, since it's now never expected to fail. Thus, any failures we see in it are worth investigating. **Additional Notes:** For context, asan (or ASan) is the "AddressSanitizer" tool, see https://github.com/google/sanitizers/wiki/AddressSanitizer for more details. **How to test the change?** Validate that the updated workflow is running on ruby-3.4.2 and that it still has the diagnostic output from asan. * DEBUG-3535 use core transport for DI to support unix domain sockets (#4426) * Composite action * Fix publish * Fix doc step with branch * Dependency inject logger (partial) (#4432) * Add 2.12.0 to CHANGELOG.md * Bump version 2.11.0 to 2.12.0 * Update lockfiles for release 2.12.0 --------- Co-authored-by: TonyCTHsu <[email protected]> Co-authored-by: Yury Lebedev <[email protected]> Co-authored-by: y9v <[email protected]> Co-authored-by: Yury Lebedev <[email protected]> Co-authored-by: Ivo Anjo <[email protected]> Co-authored-by: Oleg Pudeyev <[email protected]>
1 parent 06f110a commit 54efa06

File tree

717 files changed

+4649
-1194
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

717 files changed

+4649
-1194
lines changed

‎.github/actions/build-test/action.yml‎

Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
name: 'Build and Test'
2+
3+
# TODO: Improve description
4+
description: 'Configure test suite in batches'
5+
6+
inputs:
7+
alias:
8+
description: 'Runtime alias'
9+
required: true
10+
container-id:
11+
description: 'Container Identifier'
12+
required: true
13+
14+
runs:
15+
using: "composite"
16+
steps:
17+
- name: Configure Git
18+
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
19+
shell: bash
20+
21+
- run: bundle exec rake github:run_batch_build
22+
shell: bash
23+
24+
- name: Configure RSpec
25+
run: ln -s .rspec-local.example .rspec-local
26+
shell: bash
27+
28+
- run: bundle exec rake github:run_batch_tests
29+
shell: bash
30+
env:
31+
COVERAGE_DIR: coverage/versions/${{ inputs.alias }}/${{ inputs.container-id }}
32+
33+
- name: Debug with SSH connection
34+
if: ${{ failure() && runner.debug == '1' }}
35+
uses: mxschmitt/action-tmate@e5c7151931ca95bad1c6f4190c730ecf8c7dde48 # v3.19.0
36+
with:
37+
limit-access-to-actor: true
38+
# This mode will wait at the end of the job for a user to connect and then to terminate the tmate session.
39+
# If no user has connected within 10 minutes after the post-job step started,
40+
# it will terminate the tmate session and quit gracefully.
41+
detached: true
42+
43+
- name: Validate test agent data
44+
if: ${{ !cancelled() }}
45+
run: ruby .github/scripts/test_agent_check.rb
46+
shell: bash
47+
48+
- name: Upload junit reports
49+
if: ${{ !cancelled() }}
50+
uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
51+
with:
52+
name: junit-${{ inputs.alias }}-${{ inputs.container-id }}
53+
path: tmp/rspec/*.xml
54+
55+
- name: Upload coverage data
56+
uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
57+
with:
58+
name: coverage-${{ inputs.alias }}-${{ inputs.container-id }}
59+
path: coverage
60+
include-hidden-files: true # Coverage data generated by SimpleCov are hidden

‎.github/actions/bundle-restore/action.yml‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
name: 'Restore Bundle from cache'
2+
3+
# TODO: Improve description
4+
description: 'Restore the bundle from cache'
5+
6+
inputs:
7+
lockfile-name:
8+
description: 'Name of the lockfile artifact to download'
9+
required: true
10+
cache-key:
11+
description: 'Cache key for bundle'
12+
required: true
13+
14+
runs:
15+
using: composite
16+
steps:
17+
- name: Download lockfile
18+
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
19+
with:
20+
name: ${{ inputs.lockfile-name }}
21+
22+
- name: Restore cache
23+
uses: actions/cache/restore@1bd1e32a3bdc45362d1e726936510720a7c30a57 # v4.2.0
24+
id: restore-cache
25+
with:
26+
key: "${{ inputs.cache-key }}"
27+
path: "/usr/local/bundle"
28+
29+
- run: bundle check
30+
shell: bash

‎.github/forced-tests-list.json‎

Lines changed: 1 addition & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,3 @@
11
{
2-
"AGENT_NOT_SUPPORTING_SPAN_EVENTS":
3-
[
4-
"tests/test_span_events.py"
5-
],
6-
"PARAMETRIC":
7-
[
8-
"tests/parametric/test_span_events.py"
9-
],
10-
"DEFAULT": [
11-
"tests/test_graphql.py"
12-
]
2+
133
}

‎.github/workflows/_unit_test.yml‎

Lines changed: 100 additions & 58 deletions
Original file line numberDiff line numberDiff line change
@@ -27,6 +27,7 @@ jobs:
2727
name: batch
2828
outputs:
2929
batches: "${{ steps.set-batches.outputs.batches }}"
30+
misc: "${{ steps.set-batches.outputs.misc }}"
3031
cache-key: "${{ steps.restore-cache.outputs.cache-primary-key }}"
3132
lockfile: "${{ steps.lockfile.outputs.lockfile }}"
3233
container:
@@ -60,18 +61,47 @@ jobs:
6061
- id: set-batches
6162
name: Distribute tasks into batches
6263
run: |
63-
batches_json=$(bundle exec rake github:generate_batches)
64-
echo "$batches_json" | ruby -rjson -e 'puts JSON.pretty_generate(JSON.parse(STDIN.read))'
65-
echo "batches=$batches_json" >> $GITHUB_OUTPUT
64+
data=$(bundle exec rake github:generate_batches)
65+
echo "$data" | ruby -rjson -e 'puts JSON.pretty_generate(JSON.parse(STDIN.read))'
66+
67+
# Extract each key and set it as a separate output
68+
batches_data=$(echo "$data" | ruby -rjson -e 'puts JSON.parse(STDIN.read)["batches"].to_json')
69+
misc_data=$(echo "$data" | ruby -rjson -e 'puts JSON.parse(STDIN.read)["misc"].to_json')
70+
71+
echo "batches=$batches_data" >> $GITHUB_OUTPUT
72+
echo "misc=$misc_data" >> $GITHUB_OUTPUT
6673
- name: Generate batch summary
6774
run: bundle exec rake github:generate_batch_summary
6875
env:
6976
batches_json: "${{ steps.set-batches.outputs.batches }}"
70-
build-test:
77+
78+
# `Initialize containers` step becomes quite heavily when many services are used.
79+
#
80+
# The job can failed with timeout because it takes a long time pulling the image or waiting for the service to be ready).
81+
#
82+
# `build-test-standard` job is used to run most of our tasks and configured with the following services:
83+
# - mysql
84+
# - postgres
85+
# - redis
86+
#
87+
# `build-test-misc` job is extracted to run specific tasks that requires the following services:
88+
# - elasticsearch
89+
# - memcached
90+
# - mongodb
91+
# - opensearch
92+
# - presto
93+
#
94+
# Benefit of this optimization:
95+
# - Unnecessary services are not started
96+
# - Reduce the overhead for pulling images
97+
# - Improve reliability for starting dependent services
98+
#
99+
# In the future, we could consider to extract jobs to run database dependent tasks to optimize the workflow.
100+
build-test-standard:
71101
needs:
72102
- batch
73103
runs-on: ubuntu-24.04
74-
name: build & test [${{ matrix.batch }}]
104+
name: build & test (standard) [${{ matrix.batch }}]
75105
timeout-minutes: 30
76106
env:
77107
BATCHED_TASKS: "${{ toJSON(matrix.tasks) }}"
@@ -86,13 +116,7 @@ jobs:
86116
DD_REMOTE_CONFIGURATION_ENABLED: 'false'
87117
TEST_POSTGRES_HOST: postgres
88118
TEST_REDIS_HOST: redis
89-
TEST_ELASTICSEARCH_HOST: elasticsearch
90-
TEST_MEMCACHED_HOST: memcached
91-
TEST_MONGODB_HOST: mongodb
92119
TEST_MYSQL_HOST: mysql
93-
TEST_OPENSEARCH_HOST: opensearch
94-
TEST_OPENSEARCH_PORT: '9200'
95-
TEST_PRESTO_HOST: presto
96120
DD_AGENT_HOST: agent
97121
DD_TRACE_AGENT_PORT: '9126'
98122
DATADOG_GEM_CI: 'true'
@@ -108,6 +132,12 @@ jobs:
108132
DD_POOL_TRACE_CHECK_FAILURES: 'true'
109133
DD_DISABLE_ERROR_RESPONSES: 'true'
110134
ENABLED_CHECKS: trace_content_length,trace_stall,meta_tracer_version_header,trace_count_header,trace_peer_service,trace_dd_service
135+
mysql:
136+
image: ghcr.io/datadog/images-rb/services/mysql:8.0
137+
env:
138+
MYSQL_ROOT_PASSWORD: root
139+
MYSQL_PASSWORD: mysql
140+
MYSQL_USER: mysql
111141
postgres:
112142
image: ghcr.io/datadog/images-rb/services/postgres:9.6
113143
env:
@@ -116,6 +146,57 @@ jobs:
116146
POSTGRES_DB: postgres
117147
redis:
118148
image: ghcr.io/datadog/images-rb/services/redis:6.2
149+
steps:
150+
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
151+
- name: Restore bundle cache
152+
uses: ./.github/actions/bundle-restore
153+
with:
154+
lockfile-name: ${{ needs.batch.outputs.lockfile }}
155+
cache-key: ${{ needs.batch.outputs.cache-key }}
156+
- name: Build & Test
157+
uses: ./.github/actions/build-test
158+
with:
159+
alias: ${{ inputs.alias }}
160+
container-id: "standard-${{ matrix.batch }}"
161+
162+
build-test-misc:
163+
needs:
164+
- batch
165+
runs-on: ubuntu-24.04
166+
name: build & test (misc) [${{ matrix.batch }}]
167+
timeout-minutes: 30
168+
env:
169+
BATCHED_TASKS: "${{ toJSON(matrix.tasks) }}"
170+
strategy:
171+
fail-fast: false
172+
matrix:
173+
include: "${{ fromJson(needs.batch.outputs.misc).include }}"
174+
container:
175+
image: ghcr.io/datadog/images-rb/engines/${{ inputs.engine }}:${{ inputs.version }}
176+
env:
177+
DD_INSTRUMENTATION_TELEMETRY_ENABLED: 'false'
178+
DD_REMOTE_CONFIGURATION_ENABLED: 'false'
179+
TEST_ELASTICSEARCH_HOST: elasticsearch
180+
TEST_MEMCACHED_HOST: memcached
181+
TEST_MONGODB_HOST: mongodb
182+
TEST_OPENSEARCH_HOST: opensearch
183+
TEST_OPENSEARCH_PORT: '9200'
184+
TEST_PRESTO_HOST: presto
185+
DD_AGENT_HOST: agent
186+
DD_TRACE_AGENT_PORT: '9126'
187+
DATADOG_GEM_CI: 'true'
188+
TEST_DATADOG_INTEGRATION: '1'
189+
JRUBY_OPTS: "--dev" # Faster JVM startup: https://github.com/jruby/jruby/wiki/Improving-startup-time#use-the---dev-flag
190+
services:
191+
agent:
192+
image: ghcr.io/datadog/dd-apm-test-agent/ddapm-test-agent:v1.18.0
193+
env:
194+
LOG_LEVEL: DEBUG
195+
TRACE_LANGUAGE: ruby
196+
PORT: '9126'
197+
DD_POOL_TRACE_CHECK_FAILURES: 'true'
198+
DD_DISABLE_ERROR_RESPONSES: 'true'
199+
ENABLED_CHECKS: trace_content_length,trace_stall,meta_tracer_version_header,trace_count_header,trace_peer_service,trace_dd_service
119200
elasticsearch:
120201
image: ghcr.io/datadog/images-rb/services/elasticsearch:8.1.3
121202
env:
@@ -138,54 +219,15 @@ jobs:
138219
cluster.routing.allocation.disk.threshold_enabled: 'false'
139220
presto:
140221
image: ghcr.io/datadog/images-rb/services/starburstdata/presto:332-e.9
141-
mysql:
142-
image: ghcr.io/datadog/images-rb/services/mysql:8.0
143-
env:
144-
MYSQL_ROOT_PASSWORD: root
145-
MYSQL_PASSWORD: mysql
146-
MYSQL_USER: mysql
147222
steps:
148223
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
149-
- name: Configure Git
150-
run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
151-
- name: Download lockfile
152-
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
224+
- name: Restore bundle cache
225+
uses: ./.github/actions/bundle-restore
153226
with:
154-
name: lockfile-${{ inputs.alias }}-${{ github.run_id }}
155-
- name: Restore cache
156-
uses: actions/cache/restore@1bd1e32a3bdc45362d1e726936510720a7c30a57 # v4.2.0
157-
id: restore-cache
158-
with:
159-
key: "${{ needs.batch.outputs.cache-key }}"
160-
path: "/usr/local/bundle"
161-
- run: bundle check || bundle install
162-
- run: bundle exec rake github:run_batch_build
163-
- name: Configure RSpec
164-
run: ln -s .rspec-local.example .rspec-local
165-
- run: bundle exec rake github:run_batch_tests
166-
env:
167-
COVERAGE_DIR: coverage/versions/${{ inputs.alias }}/${{ matrix.batch }}
168-
- name: Debug with SSH connection
169-
if: ${{ failure() && runner.debug == '1' }}
170-
uses: mxschmitt/action-tmate@e5c7151931ca95bad1c6f4190c730ecf8c7dde48 # v3.19.0
171-
with:
172-
limit-access-to-actor: true
173-
# This mode will wait at the end of the job for a user to connect and then to terminate the tmate session.
174-
# If no user has connected within 10 minutes after the post-job step started,
175-
# it will terminate the tmate session and quit gracefully.
176-
detached: true
177-
- name: Validate test agent data
178-
if: ${{ !cancelled() }}
179-
run: ruby .github/scripts/test_agent_check.rb
180-
- name: Upload junit reports
181-
if: ${{ !cancelled() }}
182-
uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
183-
with:
184-
name: junit-${{ inputs.alias }}-${{ matrix.batch }}-${{ github.run_id }}
185-
path: tmp/rspec/*.xml
186-
- name: Upload coverage data
187-
uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
227+
lockfile-name: ${{ needs.batch.outputs.lockfile }}
228+
cache-key: ${{ needs.batch.outputs.cache-key }}
229+
- name: Build & Test
230+
uses: ./.github/actions/build-test
188231
with:
189-
name: coverage-${{ inputs.alias }}-${{ matrix.batch }}-${{ github.run_id }}
190-
path: coverage
191-
include-hidden-files: true # Coverage data generated by SimpleCov are hidden
232+
alias: ${{ inputs.alias }}
233+
container-id: "misc-${{ matrix.batch }}"

‎.github/workflows/publish.yml‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -126,6 +126,7 @@ jobs:
126126
- rubygems-release
127127
env:
128128
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
129+
GH_REPO: ${{ github.repository }}
129130
GEM_VERSION: ${{ needs.verify-checks.outputs.version }}
130131
permissions:
131132
contents: write
@@ -153,6 +154,10 @@ jobs:
153154
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
154155
with:
155156
fetch-depth: 0
157+
- name: Set up Ruby
158+
uses: ruby/setup-ruby@8388f20e6a9c43cd241131b678469a9f89579f37 # v1.216.0
159+
with:
160+
ruby-version: '3.3.7'
156161
- run: bundle install
157162
- id: next_version
158163
run: |
@@ -264,11 +269,16 @@ jobs:
264269
with:
265270
fetch-depth: 0
266271
- run: |
267-
JOB_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}/jobs/${{ github.job }}"
272+
JOB_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
273+
TEMP_BRANCH="update-document-v${GEM_VERSION}"
274+
275+
# Create and checkout a new branch from the tag
276+
git checkout -b "${TEMP_BRANCH}" "v${GEM_VERSION}"
277+
git push origin "${TEMP_BRANCH}"
268278
269279
gh pr create \
270280
--base release \
271-
--head master \
281+
--head "${TEMP_BRANCH}" \
272282
--title "Update document v${GEM_VERSION}" \
273283
--body "This is an auto-generated PR to update documentation from [here](${JOB_URL}). Please merge (with a merge commit) when ready." \
274284
--label "docs"

‎.github/workflows/test-memory-leaks.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,9 +24,9 @@ jobs:
2424
runs-on: ubuntu-24.04
2525
steps:
2626
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
27-
- uses: ruby/setup-ruby@8388f20e6a9c43cd241131b678469a9f89579f37 # v1.216.0
27+
- uses: ruby/setup-ruby@04c2e2b8e83fbca87e4c7ea9c16e82a1c029c074 # Adds 3.4-asan builds
2828
with:
29-
ruby-version: asan
29+
ruby-version: 3.4-asan
3030
bundler-cache: true # runs 'bundle install' and caches installed gems automatically
3131
bundler: latest
3232
cache-version: v2 # bump this to invalidate cache

0 commit comments

Comments
 (0)