Durable Functions: orchestrations remain in Running state after activity failure (.NET 8 isolated worker)

[ansonpinhero]

Description

When using Azure Durable Functions (.NET 8 isolated worker) together with the Datadog .NET Tracer, orchestrations remain stuck in the Running state after an activity throws an exception.

Without Datadog, the orchestrator correctly transitions to Failed. With Datadog enabled, the orchestration history contains a TaskFailed event, but the orchestrator never replays and the orchestration status never updates.

Expected Behavior

Orchestrator should move from Running → Failed once the activity throws.

Actual Behavior with Datadog

Orchestrator remains stuck in Running.

History shows a TaskFailed event .

Environment

Azure Functions Runtime: .NET 8 Isolated Worker

[lucaspimentel]

Hey, @ansonpinhero. There potential issues today with using the .NET tracer to instrument Durable Functions. In particular, there are code constrains on orchestrator functions which the tracer is not aware of.

Related issues:

• Durable functions: "An invalid asynchronous invocation was detected" datadog-aas-extension#284
• Discrepancies between Application Insights and DataDog for Azure Functions #3757

Our current plan is to skip instrumenting orchestrator functions since the tracer can't comply with those code constraints. I'll keep this ticket open and post any updated.

[ansonpinhero]

Thanks for the update @lucaspimentel