From 5e67d651bfa7443903e73fb3b8b922a29f7ce459 Mon Sep 17 00:00:00 2001 From: Manuel Palenzuela Merino Date: Tue, 16 Dec 2025 14:14:35 +0100 Subject: [PATCH 1/4] Add docs for aggregate_datadog_test_events tool --- content/en/bits_ai/mcp_server/setup/_index.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/content/en/bits_ai/mcp_server/setup/_index.md b/content/en/bits_ai/mcp_server/setup/_index.md index 11d353422c5..5619fbbba6c 100644 --- a/content/en/bits_ai/mcp_server/setup/_index.md +++ b/content/en/bits_ai/mcp_server/setup/_index.md @@ -404,6 +404,14 @@ Searches Datadog [Test Optimization][25] for flaky tests and returns triage-read - Show flaky tests on branch `main` for repo `github.com/org/repo`, most recent first. - List flaky tests in the `timeout` category with high failure rate (50%+) so I can prioritize fixes. +### `aggregate_datadog_test_events` +*Toolset: **software-delivery***\ +Aggregates Datadog Test events so you can quantify reliability, performance, and execution trends across tests, suites, modules, or sessions. Supports `aggregation` functions (`count`, `avg`, `sum`, `min`, `max`, `pc50`, `pc75`, `pc90`, `pc95`, `pc99`), an optional `metric` (for non-count aggregations, such as `@duration`), `group_by` facets (for breakdowns like `@test.codeowners`, `@git.branch`, or `@test.name`), and `test_level` (defaults to `test`). + +- Failure volume by branch over the last week: `aggregation=count query=@test.status:fail group_by=@git.branch`. +- Slowest suites with percentiles: `aggregation=pc95 metric=@duration test_level=suite group_by=@test.suite`. +- Pass/fail split by owners: `aggregation=count group_by=@test.codeowners query="(@test.status:pass OR @test.status:fail)"`. + ## Context efficiency The Datadog MCP Server is optimized to provide responses in a way that AI agents get relevant context without being overloaded with unnecessary information. For example: From a6e3bc495daab929610171cd2c9149f710362b55 Mon Sep 17 00:00:00 2001 From: Manuel Palenzuela Merino Date: Tue, 16 Dec 2025 16:06:32 +0100 Subject: [PATCH 2/4] more concise --- content/en/bits_ai/mcp_server/setup/_index.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/en/bits_ai/mcp_server/setup/_index.md b/content/en/bits_ai/mcp_server/setup/_index.md index 5619fbbba6c..37472212d7e 100644 --- a/content/en/bits_ai/mcp_server/setup/_index.md +++ b/content/en/bits_ai/mcp_server/setup/_index.md @@ -398,7 +398,7 @@ Aggregates CI pipeline events to produce statistics, metrics, and grouped analyt ### `get_datadog_flaky_tests` *Toolset: **software-delivery***\ -Searches Datadog [Test Optimization][25] for flaky tests and returns triage-ready details, including failure-rate stats, flaky state/category, ownership context (`codeowners`, `services`, `module`, `suite`), branch/SHA history, CI pipeline impact, and recent test-run metadata. Supports pagination with `page_cursor` and sorting by fields such as `failure_rate` or `pipelines_failed`. +Searches Datadog [Test Optimization][25] for flaky tests and returns triage details (failure rate, category, owners, history, CI impact), with pagination and sorting. - Find active flaky tests for the checkout service owned by `@team-abc`, sorted by failure rate. - Show flaky tests on branch `main` for repo `github.com/org/repo`, most recent first. @@ -406,7 +406,7 @@ Searches Datadog [Test Optimization][25] for flaky tests and returns triage-read ### `aggregate_datadog_test_events` *Toolset: **software-delivery***\ -Aggregates Datadog Test events so you can quantify reliability, performance, and execution trends across tests, suites, modules, or sessions. Supports `aggregation` functions (`count`, `avg`, `sum`, `min`, `max`, `pc50`, `pc75`, `pc90`, `pc95`, `pc99`), an optional `metric` (for non-count aggregations, such as `@duration`), `group_by` facets (for breakdowns like `@test.codeowners`, `@git.branch`, or `@test.name`), and `test_level` (defaults to `test`). +Aggregates Datadog Test events to quantify reliability and performance trends with aggregation functions, optional metrics, group-by facets, and configurable test levels. - Failure volume by branch over the last week: `aggregation=count query=@test.status:fail group_by=@git.branch`. - Slowest suites with percentiles: `aggregation=pc95 metric=@duration test_level=suite group_by=@test.suite`. From 2847fbcb0bf406a5754143a4467d92ebda3f3ec3 Mon Sep 17 00:00:00 2001 From: ManuelPalenzuelaDD <144776385+ManuelPalenzuelaDD@users.noreply.github.com> Date: Tue, 16 Dec 2025 16:14:39 +0100 Subject: [PATCH 3/4] Update content/en/bits_ai/mcp_server/setup/_index.md Co-authored-by: Joe Peeples --- content/en/bits_ai/mcp_server/setup/_index.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/en/bits_ai/mcp_server/setup/_index.md b/content/en/bits_ai/mcp_server/setup/_index.md index 37472212d7e..4b7f81bd031 100644 --- a/content/en/bits_ai/mcp_server/setup/_index.md +++ b/content/en/bits_ai/mcp_server/setup/_index.md @@ -408,9 +408,9 @@ Searches Datadog [Test Optimization][25] for flaky tests and returns triage deta *Toolset: **software-delivery***\ Aggregates Datadog Test events to quantify reliability and performance trends with aggregation functions, optional metrics, group-by facets, and configurable test levels. -- Failure volume by branch over the last week: `aggregation=count query=@test.status:fail group_by=@git.branch`. -- Slowest suites with percentiles: `aggregation=pc95 metric=@duration test_level=suite group_by=@test.suite`. -- Pass/fail split by owners: `aggregation=count group_by=@test.codeowners query="(@test.status:pass OR @test.status:fail)"`. +- Count the number of failed tests over the last week, grouped by branch. +- Show me the 95th-percentile duration for each test suite to identify the slowest ones. +- Count all passing and failing tests, grouped by code owners. ## Context efficiency From f232f6904000c65d4c46289b717df42a9464134a Mon Sep 17 00:00:00 2001 From: ManuelPalenzuelaDD <144776385+ManuelPalenzuelaDD@users.noreply.github.com> Date: Tue, 16 Dec 2025 17:33:42 +0100 Subject: [PATCH 4/4] Update content/en/bits_ai/mcp_server/setup/_index.md Co-authored-by: Joe Peeples --- content/en/bits_ai/mcp_server/setup/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/en/bits_ai/mcp_server/setup/_index.md b/content/en/bits_ai/mcp_server/setup/_index.md index 4b7f81bd031..1e3d8ce0053 100644 --- a/content/en/bits_ai/mcp_server/setup/_index.md +++ b/content/en/bits_ai/mcp_server/setup/_index.md @@ -406,7 +406,7 @@ Searches Datadog [Test Optimization][25] for flaky tests and returns triage deta ### `aggregate_datadog_test_events` *Toolset: **software-delivery***\ -Aggregates Datadog Test events to quantify reliability and performance trends with aggregation functions, optional metrics, group-by facets, and configurable test levels. +Aggregates Datadog Test Optimization events to quantify reliability and performance trends with aggregation functions, optional metrics, group-by facets, and configurable test levels. - Count the number of failed tests over the last week, grouped by branch. - Show me the 95th-percentile duration for each test suite to identify the slowest ones.