-
Notifications
You must be signed in to change notification settings - Fork 873
Update TransferUtilityConfig and BaseDownloadRequest to add multipartdownload config options #4120
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: feature/transfermanager
Are you sure you want to change the base?
Conversation
…t download config options
67bb12f to
1aa0ae7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds configuration options for multipart download functionality to the AWS S3 Transfer Utility. It introduces memory management controls and download strategy options as the first step toward implementing multipart download support (issue #3806).
Key changes:
- Added
MaxInMemoryPartsproperty to control memory buffering during downloads - Added
PartSizeproperty to configure download chunk sizes - Added
MultipartDownloadTypeenum to select between part-based and range-based download strategies
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| TransferUtilityConfig.cs | Adds MaxInMemoryParts property with default value of 1024 parts to control memory usage during multipart downloads |
| BaseDownloadRequest.cs | Adds MultipartDownloadType enum and properties (PartSize, MultipartDownloadType) to configure multipart download behavior |
| 19ed68ce-9f46-4e1e-a0ff-45a2b3641946.json | DevConfig file specifying patch version bump with changelog entries for the new properties |
| public int MaxInMemoryParts | ||
| { | ||
| get { return this._maxInMemoryParts; } | ||
| set | ||
| { | ||
| if (value < 1) | ||
| value = 1; |
Copilot
AI
Nov 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The setter silently modifies invalid input values instead of throwing an exception. This differs from standard .NET property validation patterns and can mask configuration errors. Consider throwing an ArgumentOutOfRangeException for values less than 1, similar to how the ConcurrentServiceRequests property's remarks indicate that invalid values are 'silently ignored' but the actual behavior is to clamp to a minimum. For consistency and clarity, either document this clamping behavior in the XML documentation or throw an exception for invalid values.
| public int MaxInMemoryParts | |
| { | |
| get { return this._maxInMemoryParts; } | |
| set | |
| { | |
| if (value < 1) | |
| value = 1; | |
| /// <remarks> | |
| /// Throws <see cref="ArgumentOutOfRangeException"/> if the value is less than 1. | |
| /// </remarks> | |
| public int MaxInMemoryParts | |
| { | |
| get { return this._maxInMemoryParts; } | |
| set | |
| { | |
| if (value < 1) | |
| throw new ArgumentOutOfRangeException(nameof(value), "MaxInMemoryParts must be greater than 0."); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
wondering what the correct procedure is. i think argumentoutorrange makes sense but im not sure why we silently updated the value for concurrentservicerequests?
| public long PartSize | ||
| { | ||
| get { return this.partSize.GetValueOrDefault(); } | ||
| set { this.partSize = value; } |
Copilot
AI
Nov 11, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The PartSize property lacks input validation. Negative or zero values could lead to unexpected behavior when used for multipart downloads. Consider adding validation in the setter to ensure the value is positive, similar to the validation pattern used in TransferUtilityConfig.MaxInMemoryParts.
| set { this.partSize = value; } | |
| set | |
| { | |
| if (value <= 0) | |
| { | |
| throw new ArgumentOutOfRangeException(nameof(PartSize), "PartSize must be a positive value."); | |
| } | |
| this.partSize = value; | |
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we dont do this for multi part upload in this area
| public long PartSize |
This change adds config options for multi part download.
Description
The three config options we are adding are
Motivation and Context
Testing
Types of changes
Checklist
License