support scrub http.url

[DeoLeung]

Description

from https://logfire.pydantic.dev/docs/how-to-guides/scrubbing/#keep-sensitive-data-out-of-urls it seems http.url won't be scrubbed even I define them in extra_patterns or scrub callback

i have some external vendor service which uses tokens like http://abc.com?key=[token], instruct httpx will print the sensitive token in traces. i want to scrub them so it won't leak in my trace, will it be supported or is there any workaround for it?

[alexmojaki]

import httpx

import logfire

logfire.configure()
logfire.DEFAULT_LOGFIRE_INSTANCE.config.scrubber.SAFE_KEYS.remove('http.url')

logfire.instrument_httpx()

httpx.get('https://httpbin.org/get?secret=foo')

• Removing keys from this whitelist should be better supported, this is an undocumented hack
• We should treat URLs as structured data similar to JSON, parsing the query params and scrubbing within so that everything 'just works'

[DeoLeung]

using the above works but still another problem. the message may leak it :(

[alexmojaki]

OK, more hacky, use a custom head sampler to replace the attribute before the span starts:

import httpx
from opentelemetry.sdk.trace.sampling import ALWAYS_ON, Sampler

import logfire


class MySampler(Sampler):
    def should_sample(self, parent_context, trace_id, name, kind, attributes, *args, **kwargs):
        attributes = dict(attributes or {})
        attributes['http.url'] = 'REDACTED'  # TODO scrub params here
        return ALWAYS_ON.should_sample(parent_context, trace_id, name, kind, attributes, *args, **kwargs)

    def get_description(self):
        return 'MySampler'


logfire.configure(sampling=logfire.SamplingOptions(head=MySampler()))

logfire.instrument_httpx()

httpx.get('https://httpbin.org/get?secret=foo')

[DeoLeung]

using the sampler approach works

class DispatcherSampler(Sampler):
  def should_sample(
    self, parent_context, trace_id, name, kind, attributes, *args, **kwargs
  ):
    attributes = dict(attributes or {})
    if url := attributes.get('http.url', ''):
      attributes['http.url'] = URL_PATTERN.sub('key=[REDACTED]', url)
    return ALWAYS_ON.should_sample(
      parent_context, trace_id, name, kind, attributes, *args, **kwargs
    )

  def get_description(self):
    return 'DispatcherSampler'

and in the meantime could remove check in scrubbing and safe key, thanks :)

[DeoLeung]

we observe another case the key is not scrubbed in fastapi span

i checked the sampler, it did replace the value, but fastapi instrument ignore it

[alexmojaki]

That part of the UI is just rendering whatever's in attributes, so you should be able to see the data there. I'm guessing it's in url.query.

[DeoLeung]

it's in the http.url :(

[alexmojaki]

i checked the sampler, it did replace the value, but fastapi instrument ignore it

what does this mean? why do you say it replaced the value?

[DeoLeung]

my goal is not to leak sensitive information in the url query

using the sampler approach, we succeeded in replacing the value with [redacted] in httpx instrument
for fastapi instrument, we saw the http.url got redacted in sampler, but the ui still showing them.
maybe the instrument_fastapi does something else ignoring the redacted http.url ?

[alexmojaki]

Sigh, the OTel library sets the attributes again after the span starts.

For now as a workaround I think you have remove both http.url and logfire.msg from the safe keys, then use a scrubbing callback to only actually scrub logfire.msg when it contains the key.