Fix secrets redaction for CSI volumes to avoid state corruption #27176
+49
−11
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In #12583 we changed the serialization code for CSI volumes so that we were using the extension method we use for topology and nodes. This reduces an enormous amount of boilerplate code, but we introduced a state store corruption bug in the process. The extension method sanitizes the volume without copying it, so a read of the volume (such as getting an event from the event stream) can cause the volume's secrets to be redacted in subsequent requests to publish or mount the volume.
Move the sanitization code into a testable method on the volume, and add a copy to the method.
Ref: #12583
Fixes: #26766
Testing & Reproduction steps
In addition to the unit test here, you can reproduce this behavior with the hostpath plugin demo. Comment out the steps to claim the volume. Run the
run.shscript and examine the allocation logs for the plugin. You'll see the unredacted secrets there. Then read from the event stream vianomad operator api "/v1/event/stream?topic=CSIVolume". Uncomment the steps to claim the volume and run the workload. Examine the allocation logs for the plugin again and you'll see the secret is being redacted. With this patch it works as expected.Contributor Checklist
changelog entry using the
make clcommand.ensure regressions will be caught.
and job configuration, please update the Nomad product documentation, which is stored in the
web-unified-docsrepo. Refer to theweb-unified-docscontributor guide for docs guidelines.Please also consider whether the change requires notes within the upgrade
guide. If you would like help with the docs, tag the
nomad-docsteam in this PR.Reviewer Checklist
backporting document.
in the majority of situations. The main exceptions are long-lived feature branches or merges where
history should be preserved.
within the public repository.
Changes to Security Controls
Are there any changes to security controls (access controls, encryption, logging) in this pull request? If so, explain.