False positives in Datadog Error Tracking for 4xx errors #91
Description
Prerequisites
- I have written a descriptive issue title
- I have searched existing issues to ensure the bug has not already been reported
Fastify version
4.x.x
Plugin version
0.9.4
Node.js version
22.x
Operating system
macOS
Operating system version (i.e. 20.04, 11.3, 10)
N/A
Description
Datadog’s Error Tracking treats any span exception event as a bug to fix. The fastify/otel currently calls span.recordException() for all errors thrown in route handlers, including non-bug errors (e.g., invalid input, auth, not-found). As a result, Datadog surfaces these 4xx cases as bugs to fix, resulting in noisy false positives.
app.get('/book/:id', async (req, reply) => {
const book = await findBook(req.params.id)
if (!book) {
throw new NotFoundError('book not found')
}
return { book }
})
app.setErrorHandler((err, _req, reply) => {
reply.status(err.statusCode || 500).send({ message: err.message })
})Link to code that reproduces the bug
N/A
Expected Behavior
One of:
- fastify/otel stop calling
span.recordExceptionfor thrown errors in handlers, because fastify/otel lacks information to decide if the error should be recorded. Instead app itself need to record the error. - Won't fix. Datadog error tracking have false assumption that all otel exceptions in spans are errors.