[BUG]: DeleteFiles@1 Failing With JavaScript heap out of memory

[Stuart-Wilcox]

New issue checklist

• I searched for existing GitHub issues
• I read pipeline troubleshooting guide
• I checked how to collect logs

Task name

DeleteFiles

Task version

1.255.0

Issue Description

Overview

The task DeleteFiles@1 is failing my repo with a JavaScript heap out of memory error. This started happening when running this task in my pipeline after npm dependencies were installed for a very large monorepo.

I believe the culprit is using tl.find on the entire sourceDirectory, then doing pattern matching based on that complete result. I have created a PR to update the implementation to use fast-glob: #21385

Environment type (Please select at least one enviroment where you face this issue)

• Self-Hosted
• Microsoft Hosted
• VMSS Pool
• Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Ubuntu 22

Relevant log output

Starting: Test FI Delete
==============================================================================
Task         : Delete files
Description  : Delete folders, or files matching a pattern
Version      : 1.255.0
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/delete-files
==============================================================================

<--- Last few GCs --->

[12433:0x13cb5da0]   436371 ms: Mark-Compact 4047.5 (4134.0) -> 4031.8 (4134.5) MB, 692.71 / 0.00 ms  (average mu = 0.155, current mu = 0.096) allocation failure; scavenge might not succeed
[12433:0x13cb5da0]   437115 ms: Mark-Compact 4048.0 (4134.5) -> 4032.5 (4135.0) MB, 679.77 / 0.00 ms  (average mu = 0.122, current mu = 0.086) allocation failure; scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----

 1: 0xb8cf03 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
 2: 0xf060d0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
 3: 0xf063b7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
 4: 0x1118005  [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
 5: 0x1118594 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
 6: 0x112f484 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::internal::GarbageCollectionReason, char const*) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
 7: 0x112fc9c v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
 8: 0x1105fa1 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
 9: 0x1107135 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
10: 0x10e4786 v8::internal::Factory::NewFillerObject(int, v8::internal::AllocationAlignment, v8::internal::AllocationType, v8::internal::AllocationOrigin) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
11: 0x15402c6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node]
12: 0x7f8141e99ef6 
##[error]Exit code 134 returned from process: file name '/usr/local/vss-agent/4.261.0/externals/node20_1/bin/node', arguments '"/mnt/vss/_work/_tasks/DeleteFiles_b7e8b412-0437-4065-9371-edc5881de25b/1.255.0/deletefiles.js"'.
Finishing: Test FI Delete

Full task logs with system.debug enabled

 [REPLACE THIS WITH YOUR INFORMATION] 

Repro steps

[rishabhmalikMS]

Hi @Stuart-Wilcox

Thank you for reporting this issue and providing the detailed error logs. We've been investigating the memory optimization for tl.find() in the Azure Pipelines Task Library, and your feedback is valuable for our analysis.

Questions to Help Us Investigate

To better understand your specific scenario and validate our optimization approach, could you help us with the following information?

1. Repository Scale

• Approximately how many files/folders are in your monorepo?
• What's the estimated total size of the repository?
• Are there any particularly deep directory structures (e.g., node_modules with many nested levels)?

2. Timing & Behavior

• Roughly how long did the task run before the heap out of memory error occurred?
• Was this a gradual process (task running for minutes then failing) or did it fail quickly/immediately?
• Do you see any pattern in when this started happening (specific commit, dependency changes, etc.)?

3. Environment Details

• Are you running this in a Docker container or directly on the Ubuntu 22 host?
• What's the memory limit/allocation for your build agent?
• Have you tried increasing the Node.js heap size as a temporary workaround (e.g., --max-old-space-size=8192)?

Our Analysis on fast-glob vs tl.find()
We've actually conducted extensive testing on this approach and found some interesting results:

Memory Usage Comparison (for large directory structures):

• fast-glob: ~3-4x memory usage (due to internal globbing and pattern matching overhead)