GODRIVER-3454 Support custom AWS credential provider. #2228
Conversation
🧪 Performance ResultsCommit SHA: 4fea597The following benchmark tests for version 6903c480cb08c400079f4a12 had statistically significant changes (i.e., |z-score| > 1.96):
For a comprehensive view of all microbenchmark results for this PR's commit, please check out the Evergreen perf task for this patch. |
API Change Report./v2/mongo/optionscompatible changes(*AutoEncryptionOptions).SetCredentialProviders: added ./v2/x/mongo/drivercompatible changesCred.AwsCredentialsProvider: added ./v2/x/mongo/driver/authincompatible changesMongoDBAWSAuthenticator: old is comparable, new is not ./v2/x/mongo/driver/mongocrypt/optionscompatible changes(*MongoCryptOptions).SetCredentialProviders: added |
qingyang-hu
force-pushed
the
godriver3615
branch
11 times, most recently
from
117f461 to
a1b8f64
Compare
qingyang-hu
force-pushed
the
godriver3615
branch
from
a1b8f64 to
795b157
Compare
| }) | ||
| } | ||
|
|
||
| func TestCustomAwsCredentialsProse(t *testing.T) { |
There was a problem hiding this comment.
Put this aside from TestClientSideEncryptionProse so Setenv() can be used without being influenced by t.Parallel().
qingyang-hu
requested review from
Copilot,
matthewdale and
prestonvasquez
and removed request for
prestonvasquez
There was a problem hiding this comment.
Pull Request Overview
This PR adds support for custom AWS credential providers, enabling applications to supply their own AWS credential retrieval logic for both authentication and client-side encryption. The implementation allows users to provide custom credential providers as a callback function that returns AWS credentials on demand.
Key changes:
- Added
AwsCredentialsProvidercallback field to credential options for connection authentication - Extended client-side encryption options to support custom credential providers via
CredentialProvidersmap - Refactored internal credential provider interface to use context-aware
Retrieve(context.Context)method consistently across all provider implementations
Reviewed Changes
Copilot reviewed 25 out of 25 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
mongo/options/clientoptions.go |
Added AwsCredentialsProvider field and Credentials type to support custom AWS credential callbacks |
mongo/options/clientencryptionoptions.go |
Added CredentialProviders map and setter for custom provider configuration |
mongo/options/autoencryptionoptions.go |
Added CredentialProviders field and setter for auto-encryption provider configuration |
x/mongo/driver/topology/topology_options.go |
Converted user-provided credential provider to internal format during credential conversion |
x/mongo/driver/driver.go |
Added AwsCredentialsProvider field to Cred struct for internal provider storage |
x/mongo/driver/auth/mongodbaws.go |
Integrated custom credential provider into AWS authentication flow |
x/mongo/driver/mongocrypt/options/mongocrypt_options.go |
Added CredentialProviders field to MongoCrypt options |
x/mongo/driver/mongocrypt/mongocrypt.go |
Wired custom credential providers into MongoCrypt initialization |
mongo/client.go |
Converted credential providers for auto-encryption setup |
mongo/client_encryption.go |
Converted credential providers for client encryption setup |
internal/credproviders/aws_provider.go |
New provider implementation that wraps custom AWS credential callbacks |
internal/aws/types.go |
Added public Credentials type for custom provider return values |
internal/aws/credentials/credentials.go |
Unified provider interface to use context-aware Retrieve(context.Context) method |
internal/credproviders/*.go |
Updated all provider implementations to use context-aware retrieval |
x/mongo/driver/auth/mongodbaws_test.go |
Added tests for custom credential provider behavior |
internal/integration/client_side_encryption_prose_test.go |
Added integration tests for custom credential provider scenarios |
mongo/client_examples_test.go |
Added example demonstrating custom credential provider usage |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if a.credentials == nil || a.credentials.ExpirationCallback == nil { | ||
| return true | ||
| } |
There was a problem hiding this comment.
The IsExpired logic returns true when credentials are nil, which means the provider will always be considered expired before the first retrieval. This forces an immediate retrieval, but if ExpirationCallback is also nil after the first retrieval, it will always return true, causing credentials to be re-fetched on every call. Consider returning a.credentials == nil when ExpirationCallback is nil to avoid unnecessary re-fetching after initial retrieval.
| if a.credentials == nil || a.credentials.ExpirationCallback == nil { | |
| return true | |
| } | |
| if a.credentials == nil { | |
| return true | |
| } | |
| if a.credentials.ExpirationCallback == nil { | |
| return false | |
| } |
| providers[k] = &credproviders.AwsProvider{ | ||
| Provider: func(ctx context.Context) (aws.Credentials, error) { | ||
| var creds aws.Credentials | ||
| c, err := fn(ctx) | ||
| if err != nil { | ||
| return creds, err | ||
| } | ||
| creds.AccessKeyID = c.AccessKeyID | ||
| creds.SecretAccessKey = c.SecretAccessKey | ||
| creds.SessionToken = c.SessionToken | ||
| creds.ExpirationCallback = c.ExpirationCallback | ||
| return creds, nil | ||
| }, | ||
| } |
There was a problem hiding this comment.
The closure over fn in the loop may capture the wrong value if the loop variable is reused. While this loop only processes the 'aws' key specifically, consider using a function parameter or declaring fn within the block to ensure correct closure capture: provider := fn before the closure.
| providers[k] = &credproviders.AwsProvider{ | ||
| Provider: func(ctx context.Context) (aws.Credentials, error) { | ||
| var creds aws.Credentials | ||
| c, err := fn(ctx) | ||
| if err != nil { | ||
| return creds, err | ||
| } | ||
| creds.AccessKeyID = c.AccessKeyID | ||
| creds.SecretAccessKey = c.SecretAccessKey | ||
| creds.SessionToken = c.SessionToken | ||
| creds.ExpirationCallback = c.ExpirationCallback | ||
| return creds, nil | ||
| }, | ||
| } |
There was a problem hiding this comment.
The closure over fn in the loop may capture the wrong value if the loop variable is reused. While this loop only processes the 'aws' key specifically, consider using a function parameter or declaring fn within the block to ensure correct closure capture: provider := fn before the closure.
qingyang-hu
force-pushed
the
godriver3615
branch
from
f64ee4a to
1a5bc81
Compare
qingyang-hu
force-pushed
the
godriver3615
branch
from
1a5bc81 to
4fea597
Compare
| } | ||
|
|
||
| // CredentialsProvider is the function type that returns AWS credentials. | ||
| type CredentialsProvider func(context.Context) (Credentials, error) |
There was a problem hiding this comment.
Thanks for this work @qingyang-hu! Before committing to a detailed review, I'd like to align with the team on the overall approach for providing custom credentials by clarifying that there are (at least) two approaches:
- The callback approach suggested in this PR
- The submodule to use the official aws-sdk-go-v2 as prescribed in GODRIVER-3567 and POC’d in PR #2057
Here is how I anticipate folks would use the submodule approach (see here for required POC updates):
// Custom credentials provider using the aws-sdk-go-v2
myProvider := aws.CredentialsProviderFunc(func(ctx context.Context) (aws.Credentials, error) {
return aws.Credentials{
AccessKeyID: "USER_ACCESS_KEY",
SecretAccessKey: "USER_SECRET_KEY",
SessionToken: "USER_SESSION_TOKEN",
Source: "CustomProvider",
}, nil
})
// Load AWS config with custom credentials provider using the aws-sdk-go-v2
awsCfg, _ := config.LoadDefaultConfig(
ctx,
config.WithCredentialsProvider(myProvider))
opts := options.Client().ApplyURI(uri)
// Extend options to include awsConfig using mongoaws
opts = mongoawsv2.WithAWSConfig(opts, awsConfig)
// Use with MongoDB
client, _ := mongo.Connect(opts)The pros of the submodule solution:
- Ideal for power users
- Zero AWS dependencies in the Go Driver while still getting official AWS SDK support
- The solution is future proof in that we can add mongoawsv3, mongoawsv4, etc. whenever needed
- Much better user experience than having to define an abstract credential object that could apply to any provider
- We can deprecate and no longer add to the AWS legacy code.
The Cons of the submodule solution:
- We have to maintain the legacy behavior until v3
- Requires an extra import when working with AWS
A potential concern is that the opaque any return type reduces compile-time type safety, but this is validated at connection time via mongo.Connect(), which is acceptable.
If we think custom credentials are rarely needed (power users only), then I suggest prioritizing GODRIVER-3567 (submodule approach) to avoid committing AWS-specific types to the stable API. Power users who need custom credentials likely already use AWS SDK v2 in their applications, so the dependency is acceptable. This prevents long-term technical debt from maintaining options.Credentials.
Otherwise, if credentials have broader appeal or users want to avoid AWS SDK v2 dependency, then this PR is acceptable imo. However, if we merge, I suggest we prioritize implementing GODRIVER-3567 afterward and clearly document it as the preferred approach for users already using AWS SDK v2 by deprecating options.Credentials.
IIUC, the SetCredentialProviders portion for CSFLE provides value regardless of which path we choose for connection authentication.
CC: @matthewdale
GODRIVER-3454
GODRIVER-3615
Summary
Support custom AWS credential provider.
Background & Motivation