Skip to content

Add ability to configure Kafka producers individually #357

@jbeemster

Description

@jbeemster

Currently we only allow a single producerConf for both Kafka Producers for the raw and bad output streams. This means we end up dynamically inferring things like client.id which are then set to producer-1 and producer-2 (and which renders the Stream Lineage view in Confluent Cloud effectively useless.

It also means permissions for multiple topics need to be shared rather than being individual.

All other Kafka applications enforce segmented configuration for each producer + consumer and the Collector should follow suit.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions