add case sensitivity check to download directory command

[philasmar]

Description

A security fix has been applied to the Transfer Utility Download Directory command which now checks if the directory is case-sensitive before proceeding with the download. This prevents potential overwriting of files in case-insensitive file systems. If you would like to revert to the previous behavior, you can set the 'SkipDirectoryCaseSensitivityCheck' property on the 'TransferUtilityConfig' to true.

Motivation and Context

DOTNET-8392

Testing

Ran tests locally
Ran dry run

Screenshots (if appropriate)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• My code follows the code style of this project
• My change requires a change to the documentation
• I have updated the documentation accordingly
• I have read the README document
• I have added tests to cover my changes
• All new and existing tests passed

License

• I confirm that this pull request can be released under the Apache 2 license

[GarrettBeatty]

is there any way to do this where we are not repeating the logic for constructing the failure policy and creating the request.

[GarrettBeatty]

why cant we add a check in DownloadSingleFileAsync for each file?

[philasmar]

I am trying to detect a collision before actually starting the download. we have the ability to do so, so why let the download start and then have an exception thrown.

[GarrettBeatty]

current code

ExecuteAsync:
  │
  ├─ Step 1: Validate & setup directory
  │
  ├─ Step 2: List S3 objects
  │
  ├─ Step 3: Filter to actual files
  │
  ├─ Step 4: Create download resources
  │     │
  │     ├─► CheckIfKeysDifferByCase()          ← SEPARATE PHASE
  │     │     └─ for each s3Object:
  │     │          └─ await _failurePolicy.ExecuteAsync(...)  ← Async + failure policy per object
  │     │
  │     └─► ExecuteParallelDownloads()
  │           └─ for each s3Object (concurrent):
  │                └─ DownloadSingleFileAsync()
  │                      ├─ Validate path
  │                      └─ Execute download
  │
  └─ Step 5: Build response

suggested

ExecuteAsync:
  │
  ├─ Step 1: Validate & setup directory
  │
  ├─ Step 2: List S3 objects
  │
  ├─ Step 3: Filter to actual files
  │
  ├─ Step 4: Create download resources
  │     │
  │     ├─► IdentifyCaseConflictingKeys()      ← SYNC, simple loop, no failure policy
  │     │     └─ Build HashSet of conflicting keys (O(n) one pass)
  │     │
  │     └─► ExecuteParallelDownloads()
  │           └─ for each s3Object (concurrent):
  │                └─ DownloadSingleFileAsync()
  │                      ├─ Validate path
  │                      ├─ ValidateCaseConflict()  ← CHECK MOVED HERE (with failure policy)
  │                      └─ Execute download
  │
  └─ Step 5: Build response