Test Stable Config support for extended configs #5357

mtoffl01 · 2025-09-26T17:21:16Z

Motivation

In Phase 1, tracer SDKs supported a shortlist of product-enablement configurations in Declarative Config (formerly “Stable Config”) files.
In Phase 2 (“extended configs”), SDKs now support all valid configurations.

These tests use DD_TAGS and DD_TRACE_PROPAGATION_STYLE as representative examples of this broader support, since they were not part of the original Phase 1 list, and they behave consistently (...mostly) across SDKs according to the Feature Parity Dashboard.

Changes

Refactor:
Moved service-targeting subtests from Test_Stable_Config_Default into a new test class, Test_Stable_Config_Rules.
Test_Stable_Config_Default now focuses on apm_configuration_default behavior, while Test_Stable_Config_Rules covers apm_configuration_rules behavior.

New tests:
Added a Test_Stable_Config_Default case that verifies DD_TAGS and DD_TRACE_PROPAGATION_STYLE can be configured via Declarative Config files (both local and fleet sources).

Added two new test_stable_configuration_origin_extended_configs tests which assert that DD_TAGS and DD_TRACE_PROPAGATION_STYLE report telemetry with an origin of either fleet_stable_config or local_stable_config when configured through Declarative Config. There are two tests with this logic: one for "good_use_case" and one for "temporary_use_case." the latter accounts for current Ruby and Python behavior, which differs slightly from other SDKs but isn’t directly relevant to the intent of this test.

Workflow

⚠️ Create your PR as draft ⚠️
Work on you PR until the CI passes
Mark it as ready for review
- Test logic is modified? -> Get a review from RFC owner.
- Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

If PR title starts with [<language>], double-check that only <language> is impacted by the change
No system-tests internal is modified. Otherwise, I have the approval from R&P team
A docker base image is modified?
- the relevant build-XXX-image label is present
A scenario is added (or removed)?
- Get a review from R&P team

vpellan · 2025-10-01T09:21:51Z

tests/parametric/test_config_consistency.py

+        [
+            (
+                "tags",
+                {"DD_TAGS": ["tag1:value1", "tag2:value2"]},


It seems that this line causes an error in libdatadog. The log we have in ruby is this:
apm_configuration_default.DD_TAGS: invalid type: sequence, expected a string at line 3 column 3 which is a libdatadog log.
libdatadog can only parse strings, which I believe is the correct behaviour as it basically acts the same as environment variables.
This test should fail on all libs that uses libdatadog but NodeJS only accepts configs defined in the RFC, and PHP and Dotnet does not run them yet, which is why only Ruby and Python fails.
The correct value should be something like

Suggested change

{"DD_TAGS": ["tag1:value1", "tag2:value2"]},

{"DD_TAGS": "tag1:value1,tag2:value2"},

I confirm I also get 2025-10-01 15:43:07.679 +02:00 [WRN] Failed to create the global configuration source with status: LibDatadogCallError and error message: apm_configuration_default.DD_TAGS: invalid type: sequence, expected a string at line 3 column 12 .. }

vpellan · 2025-10-01T09:38:51Z

tests/parametric/test_config_consistency.py

+            (
+                "tags",
+                {"DD_TAGS": ["tag1:value1", "tag2:value2"]},
+                {"dd_tags": "[tag1:value1,tag2:value2]"},


The format of the result depends on the implementation on the weblog, but most weblogs does not include the square brackets. In this state the test would still fail for them. The easiest fix would be something like:

Suggested change

{"dd_tags": "[tag1:value1,tag2:value2]"},

{

"dd_tags": "[tag1:value1,tag2:value2]"

if context.library in ["golang", "dotnet"]

else "tag1:value1,tag2:value2"

},

after running locally we seem to indeed format it as an array in dotnet but with a space in the middle 😶‍🌫️ "dd_tags": "[tag1:value1, tag2:value2]" should work for us

actually my bad, after double checking it should be :

Suggested change

{"dd_tags": "[tag1:value1,tag2:value2]"},

{

"dd_tags": "[tag1:value1,tag2:value2]"

if context.library in ["golang"]

else ['tag1:value1', 'tag2:value2']

if context.library in ["dotnet"]

else "tag1:value1,tag2:value2"

},

I know it's terrible... I wish we had a better way to check stuff, like a more dynamic way 🤔

Checking php code and they seem to simply return the input value, so if you decide to also keep the space in the input, this will be another case. To avoid having a lot of cases where the only difference will be brackets and spaces, another way to fix this would be to parse the expectation and check that the result contains each expectation entry, something like

system-tests/tests/parametric/test_otel_env_vars.py

Lines 43 to 47 in 31c3b79

tags = resp["dd_tags"]

assert "foo:bar" in tags

assert "baz:qux" in tags

assert "foo:otel_bar" not in tags

assert "baz:otel_qux" not in tags

Edit: Just saw Anna previous comment, this should also work with arrays I believe ?

github-actions · 2025-10-02T13:57:11Z

CODEOWNERS have been resolved as:

manifests/cpp.yml                                                       @DataDog/dd-trace-cpp
manifests/dotnet.yml                                                    @DataDog/apm-dotnet @DataDog/asm-dotnet
manifests/golang.yml                                                    @DataDog/dd-trace-go-guild
manifests/java.yml                                                      @DataDog/asm-java @DataDog/apm-java
manifests/nodejs.yml                                                    @DataDog/dd-trace-js
manifests/php.yml                                                       @DataDog/apm-php @DataDog/asm-php
manifests/python.yml                                                    @DataDog/apm-python @DataDog/asm-python
manifests/ruby.yml                                                      @DataDog/ruby-guild @DataDog/asm-ruby
tests/parametric/test_config_consistency.py                             @DataDog/apm-sdk-capabilities
tests/parametric/test_telemetry.py                                      @DataDog/system-tests-core @DataDog/apm-sdk-capabilities

mtoffl01 · 2025-10-09T18:29:53Z

tests/parametric/test_telemetry.py

+        context.library in ["cpp", "golang", "nodejs"],
+        reason="extended configs are not supported",
+    )
+    @bug(context.library == "ruby", reason="APMAPI-1650")


This will be removed once related ruby fix is merged (in progress)

DataDog/dd-trace-rb#4963 has been merged, this line can be removed (tested locally)

vpellan

LGTM!

vpellan · 2025-10-13T08:26:56Z

tests/parametric/test_telemetry.py

+    )
+    @bug(context.library == "ruby", reason="APMAPI-1650")
+    @irrelevant(context.library in ["java", "php", "dotnet"], reason="temporary use case for python and ruby")
+    def test_stable_configuration_origin_extended_configs_temporary_use_case(


I'll ask the reason why we don't report the tags and why we report the extract/inject config instead of the main one when only the main one is set

vpellan · 2025-10-13T08:28:19Z

tests/parametric/test_telemetry.py

+        context.library in ["cpp", "golang", "nodejs"],
+        reason="extended configs are not supported",
+    )
+    @bug(context.library == "ruby", reason="APMAPI-1650")


DataDog/dd-trace-rb#4963 has been merged, this line can be removed (tested locally)

anna-git

thank you!

cbeauchesne

From framework usage, all good. I suggest getting a review with someone familiar with the feature

mtoffl01 and others added 6 commits September 26, 2025 13:20

Introduce extended_configs test with dd_tags

0cb11db

Correct placement of annotation

35996f7

Enable test for dotnet, python; include test log msg for debugging

6de8166

remove print; expose config inside the test assertion

84e7856

Merge branch 'main' into mtoff/scfg-all-configs

b6c40e3

Use propagation_style for another input

27874b7

vpellan reviewed Oct 1, 2025

View reviewed changes

mtoffl01 added 4 commits October 1, 2025 16:03

Introduce telemetry test for extended_configs

f4b0908

change dd_tags input to exclude brackets

21a11bf

Change tags to not use brackets in telemetry test

4fcb3dc

run linter

beaa171

mtoffl01 and others added 14 commits October 2, 2025 12:36

Modify expected tags value for dotnet

286a640

change tags value for php

51021f7

Make extended_config telemetry test more verbose on failure

4c6ee6f

Merge branch 'main' into mtoff/scfg-all-configs

0b2f111

Change telemetry order to use propagation_style for fleet override

a8751a2

map tags to DD_TAGS for dotnet telemetry

f3c9b99

map tags to DD_TAGS for python telemetry

e281615

map propagation_style for python and dotnet

6401306

swap python and dotnet for telemetry mapping - propagation style

3dabf60

Add entry for php telemetry mapping

fc65384

Add entry for ruby telemetry mapping

916e25d

run liner

5ffdeb7

Use inject and extract propagation style keys for python and ruby

2648b8f

Split config test into 'default' and 'rules'

c37fcab

vpellan mentioned this pull request Oct 9, 2025

[NO-TICKET] Fix precedence for tracing.propagation_style_extract and inject when set through tracing.propagation_style DataDog/dd-trace-rb#4963

Merged

mark telemetry test with bug for ruby

f1759a6

mtoffl01 commented Oct 9, 2025

View reviewed changes

mtoffl01 marked this pull request as ready for review October 9, 2025 20:04

mtoffl01 requested review from a team as code owners October 9, 2025 20:04

mtoffl01 requested review from Anilm3, christophe-papazian, link04, manuel-alvarez-alvarez, robertpi and vlad-scherbich and removed request for a team October 9, 2025 20:04

merge main/ resolve conflicts

f0b51e7

mtoffl01 requested review from anna-git and vpellan October 10, 2025 19:16

vpellan approved these changes Oct 13, 2025

View reviewed changes

anna-git approved these changes Oct 13, 2025

View reviewed changes

Merge branch 'main' into mtoff/scfg-all-configs

9e50750

cbeauchesne approved these changes Oct 14, 2025

View reviewed changes

Merge branch 'main' into mtoff/scfg-all-configs

3041acd

mtoffl01 merged commit 59526bb into main Oct 15, 2025
404 checks passed

mtoffl01 deleted the mtoff/scfg-all-configs branch October 15, 2025 15:18

	{"DD_TAGS": ["tag1:value1", "tag2:value2"]},
	{"DD_TAGS": "tag1:value1,tag2:value2"},

-                {"dd_tags": "[tag1:value1,tag2:value2]"},
+                 {
+                    "dd_tags": "[tag1:value1,tag2:value2]"
+                    if context.library in ["golang"]
+                    else ['tag1:value1', 'tag2:value2']
+                    if context.library in ["dotnet"]
+                    else "tag1:value1,tag2:value2"
+                },

	tags = resp["dd_tags"]
	assert "foo:bar" in tags
	assert "baz:qux" in tags
	assert "foo:otel_bar" not in tags
	assert "baz:otel_qux" not in tags

Test Stable Config support for extended configs #5357

Test Stable Config support for extended configs #5357

Uh oh!

Conversation

mtoffl01 commented Sep 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Workflow

Reviewer checklist

Uh oh!

vpellan Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

anna-git Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

vpellan Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

anna-git Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anna-git Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vpellan Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mtoffl01 Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

vpellan Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

vpellan left a comment

Choose a reason for hiding this comment

Uh oh!

vpellan Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

vpellan Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

anna-git left a comment

Choose a reason for hiding this comment

Uh oh!

cbeauchesne left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mtoffl01 commented Sep 26, 2025 •

edited

Loading

anna-git Oct 1, 2025 •

edited

Loading

anna-git Oct 1, 2025 •

edited

Loading

vpellan Oct 1, 2025 •

edited

Loading

github-actions bot commented Oct 2, 2025 •

edited

Loading