OpAMP Server is treated as trusted root authority for messages, resulting in potential vulnerabilities

[jackgopack4]

The OpAMP Server is currently treated as both the distribution point for ServerToAgent messages as well as the root authoritative source for the same. These include varying categories of messages that, should an OpAMP Server be compromised, could cause unintended behavior, data loss, or in worst case, a breach of sensitive information or takeover of host machines.

Background

Existing models for distributing packages/updates already have a separate concept of “secure distribution server” and “trusted root authority.” These include, but are not limited to:

• Google Play Store - distribution server separate from root authority (Google Play signed package)
• Windows System updates - downloaded via TLS-secured endpoints but are signed/verified by the publisher/vendor
• RPM package repositories - secure connection to the distribution source happens via TLS/HTTPS but the packages themselves are signed by GPG and metadata verified via SHA256 checksum

All of these models use a secure endpoint as a “distribution server” but also have additional separation of concern for a “root authority” of the message, signed by a trusted root/vendor. From hereout I will refer to these concerns as “distribution server” and “trusted root” or “trusted root authority” for clarity.

Attack Vectors/Vulnerabilities

For example, if an OpAMP Server were to be compromised, the following types of attacks could be executed against an agent that accepts remote config and restart commands:

1. Forced restarts/DOS
   1. OpAMP Server repeatedly sends ServerToAgentCommand.Restart to the OpAMP Agent.
   2. Agent will accept these commands believing they are coming from a trusted root (the OpAMP Server), and restart repeatedly, causing a loss of telemetry/outage
2. Telemetry redirection/”stealing” data
   1. Server sends AgentRemoteConfig with undesired fields (i.e. changing OTLP Exporter endpoint to a different address, e.g. vendor1.com/v1/logs -> attacker.url/v1/logs
   2. Agent applies remote config, trusting the OpAMP Server as the trusted root, and start sending telemetry to an entirely different endpoint (which could include sensitive/customer data if the user is not properly redacting their telemetry)
3. Unintended binary update (partially mitigated by DownloadableFile.signature)
   1. Server sends PackagesAvailable with an unexpected distribution of OTel Collector
   2. If the URL/database where the OpAMP agent matches the expected signature to a given binary for a distribution is compromised, the DownloadableFile.signature could appear to be valid, so the OpAMP Agent would download and install this binary

Each of these issues stems from the OpAMP Server being trusted as both the distribution server for OpAMP messages, as well as the trusted root authority.

Mitigation/Solution

Beyond just the signature field for a DownloadableFile message, there should be a way to separate the concerns of the trusted root authority from the distribution server, and for an OpAMP Agent to determine that any message has in fact originated from this trusted root. This might include using a pre-coordinated root certificate between OpAMP Agent and OpAMP Server, or exchanging this certificate upon initial connection between the Agent and Server, but the exact implementation details of any solution are outside the scope of this issue.

If an OpAMP Server operator (or vendor) chooses to adopt the model of a separate trusted root authority for messages, they could separate concerns by signing messages to send to a particular OpAMP Agent in a separate environment from the OpAMP server itself.
Additionally, a message signing service/process could enforce schemas or field validation (e.g. “endpoint” config values must have securecorp.com as base url), and refuse to sign messages that don’t match this requirement. In this manner, an attacker gaining access to the OpAMP Server would not be able to sign messages that do not meet strict validation/type requirements.
If the OpAMP Agent has a method to validate the trusted root certificate, and that certificate is not present or is invalid, the Agent would reject this message and could immediately terminate the connection, preventing further attacks.

Current Model vs. Proposed Model

The diagram below shows the way that this model can reduce the risk of an unauthorized actor sending unintended commands to an OpAMP Agent.

[jaronoff97]

definitely in favor of this model, as it's pretty industry standard as mentioned above. I also think it would be great to make this composable with a capability where the agent declares that it requires signed messages and the server must upgrade to sign every further message. Something to that degree would ensure a smooth upgrade path and backwards compatibility.

[tigrannajaryan]

All, we are looking for community feedback on this proposal. Please feel free to comment or do a thumbsup/down.

[michel-laterman]

I think this is something we should definitely support in spec.

[jackgopack4]

currently working on a proof-of-concept in opamp-go library using custom messages/capabilities, will link a PR when it's ready for review