fingerprint: Add retry and failure config to env fingerprinters. #27161
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change introduces new optional client fingerprinter configuration fields which can be used to control how the env fingerprinters perform retries and whether errors should halt the agent startup. The retry wrapper is used by the env_aws, env_azure, env_gce, and env_digitalocean fingerprinters and is the handler for retry and error logic on the main fingerprinter. The change is backwards compatible, so running this change without any new config options results in the same behaviour as previously. - retry_interval: Specifies the time to wait between fingerprint attempts. This will default to 2 seconds. - retry_attempts: Specifies the maximum number of fingerprint retries to be made. This will default to 0 and can be set to -1 if the operator wants infinite retries. - exit_on_failure: Determines how the agent handles failure in performing the fingerprint. The change helps alleviate problems in cloud providers where a machine starts before the metadata service and endpoint is available. In this situation, Nomad times out the fingerprinter quickly and marks it as skipped, thus assuming we are not running within that environment. Operators can use the new configuration options to handle these race conditions, and wait for the metadata service to be available and respond.
15 tasks
pkazmierczak
previously approved these changes
Nov 28, 2025
Contributor
pkazmierczak
left a comment
pkazmierczak
left a comment
There was a problem hiding this comment.
LGTM! Left some minor typo-related comments, but nothing blocking.
Co-authored-by: Piotr Kazmierczak <[email protected]>
tgross
reviewed
Dec 3, 2025
|
|
||
| // Fingerprint is an optional configuration block for environment fingerprinters | ||
| // can control retry behavior and failure handling. | ||
| type Fingerprint struct { |
Member
There was a problem hiding this comment.
In non-cloud environments we'll typically see something like the following to disable all the cloud fingerprinters. This uses the deprecated options syntax. Should we add an enabled flag to this struct to implement the same behavior?
client {
options = {
"fingerprint.denylist" = "env_aws,env_gce,env_azure,env_digitalocean"
}
}
Comment on lines
+49
to
+56
| // Fingerprint executes the underlying fingerprinter with retry logic based | ||
| // on the client configuration and implements the Fingerprinter interface. | ||
| // | ||
| // If the fingerprinter fails after all retry attempts, the error from the last | ||
| // attempt is returned, unless the configuration indicates that failures should | ||
| // be skipped for this fingerprinter and the error is of the type that indicates | ||
| // an initial probe failure. | ||
| func (rw *RetryWrapper) Fingerprint(req *FingerprintRequest, resp *FingerprintResponse) error { |
Member
There was a problem hiding this comment.
I'm not sure I understand how we're differentiating between failures we need to retry (and block initial fingerprinting for) and failures because that's not the environment we're in. Does this end up blocking the first fingerprint?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change introduces new optional client fingerprinter configuration fields which can be used to control how the env fingerprinters perform retries and whether errors should halt the agent startup.
The retry wrapper is used by the env_aws, env_azure, env_gce, and env_digitalocean fingerprinters and is the handler for retry and error logic on the main fingerprinter. The change is backwards compatible, so running this change without any new config options results in the same behaviour as previously.
The change helps alleviate problems in cloud providers where a machine starts before the metadata service and endpoint is available. In this situation, Nomad timesout the fingerprinter quickly and marks it as skipped, thus assuming we are not running within that environment. Operators can use the new configuration options to handle these race conditions, and wait for the metadata service to be available and respond.
Links
Jira: https://hashicorp.atlassian.net/browse/NMD-1061
Contributor Checklist
changelog entry using the
make clcommand.ensure regressions will be caught.
and job configuration, please update the Nomad product documentation, which is stored in the
web-unified-docsrepo. Refer to theweb-unified-docscontributor guide for docs guidelines.Please also consider whether the change requires notes within the upgrade
guide. If you would like help with the docs, tag the
nomad-docsteam in this PR.Reviewer Checklist
backporting document.
in the majority of situations. The main exceptions are long-lived feature branches or merges where
history should be preserved.
within the public repository.