editoast: add trace_id to all responses

[eckter]

Part of #11478, front excluded

When opentelemetry is set, responses now contain a header such as

traceparent: 00-e535ebd1df21be08aabe3f85f8dd8d4f-c8d72478e4619cf8-01

What we call the "trace id" is what's contained between the first and second -. I assume the rest can still be relevant for some tooling, and it doesn't hurt to include a little more context.

I haven't looked into the front-end side yet.

Tested manually by looking through response headers.

[github-actions]

⚠️ API changes

This Pull Request introduces some changes in the API:

• please own it: notify or even prepare dedicated PR(s) to consumer projects

[shenriotpro]

nice

[woshilapin]

Only a general comment on the proposed solution. In OpenTelemetry, the TraceID is usually forwarded in headers, not in the main payload. It seems that this also applies for returning information to a frontend (I’ve found this discussion about the topic). So that would mean:

• modifying the gateway so it returns the correct header in responses to the client
• probably modify editoast so it returns the correct header to the gateway

Maybe Devops will have an opinion about this? (cc @Khoyo @gaetan-osrd)

[woshilapin]

On the front side, it seems to be possible to do with RTK on a per-endpoint granularity (which is exactly what we want here) with the help of transformResponse (thank you @emersion)

[eckter]

Only a general comment on the proposed solution. In OpenTelemetry, the TraceID is usually forwarded in headers, not in the main payload. It seems that this also applies for returning information to a frontend (I’ve found this discussion about the topic). So that would mean:

I don't have a strong preference. Headers feel more "clean". But the goal here isn't just correlating telemetry, it's an actual data to be included in the final response (generated PDF file).

(btw I didn't plan for this ticket to take more than a few hours)

[flomonster]

Only a general comment on the proposed solution. In OpenTelemetry, the TraceID is usually forwarded in headers, not in the main payload. It seems that this also applies for returning information to a frontend (I’ve found this discussion about the topic). So that would mean:

* modifying the gateway so it returns the correct header in responses to the client

* probably modify editoast so it returns the correct header to the gateway

Maybe Devops will have an opinion about this? (cc @Khoyo @gaetan-osrd)

I don't think the gateway changes the headers of the response. I agree that having the information in the header is a better option.

It is possible to handle otel in the frontend so the trace_id is defined by the frontend and not the backend so it can retrieve it directly (without having to add it manually in the response).

[Khoyo]

I agree, we should follow opentelemetry semantics, but as long as the frontend is not actually sending telemetry data, we probably can go either way.

[eckter]

Sounds good to me.

But I might need somme help with editoast tooling, I still struggle to navigate the libraries and traits. I don't think there's any example in the code base where we add response headers, and it doesn't seem to fit easily into the -> Result<Json<Response>> pattern.

edit, self notes for later: apparently if we return a (Headers, Json) it just somehow works. Tuples of IntoResponse do implement IntoResponse

[woshilapin]

@eckter For editoast, I believe it’s only about adding the right middleware from axum-tracing-opentelemetry. If you look here, the middleware is the one that read headers for incoming request, and populate correctly the OpenTelemetry context. In the documentation of axum-tracing-opentelemetry (in the first code example), they mention the following that should be exactly what we need.

        // include trace context as header into the response
        .layer(OtelInResponseLayer::default())

[eckter]

It's now returned in the header.

It is indeed as simple as @woshilapin said, it's now a tiny PR! I still somehow lost a lot of time there because I was testing it without enabling opentelemetry...

Note: this change affects every endpoint, not just stdcm.

(and today I learned that github automatically closes a PR if one accidentally force pushes with no commit on the branch)