TaskHost Does Not Support Critical IBuildEngine Callbacks

[SimaTian]

Summary

When MSBuild runs in multithreaded mode (-mt), tasks that are not marked as thread-safe are automatically forced to run out-of-process in a TaskHost. However, the TaskHost has significant gaps in its IBuildEngine implementation—several critical callbacks either fail with an error or throw NotImplementedException. This causes builds to fail for any task that uses these callbacks.

This is a blocking issue for multithreaded adoption where the taskhost is the fallback mechanism for unenlightened tasks

Affected Scenarios

1. Multithreaded builds (-mt mode) - Tasks fallback to TaskHost and may fail
2. MSBUILDFORCEALLTASKSOUTOFPROC=1 - Environment variable that forces all tasks out-of-proc
3. Explicit TaskHostFactory usage - User-specified out-of-proc task execution
4. Architecture/Runtime mismatch - Tasks requiring different bitness or runtime

Root Cause

The Exclusion Logic is Incomplete

Located in src/Build/Instance/TaskRegistry.cs lines 1497-1507:

// If ForceAllTasksOutOfProc is true, we will force all tasks to run in the MSBuild task host
// "EXCEPT a small well-known set of tasks that are known to depend on IBuildEngine callbacks
// as forcing those out of proc would be just setting them up for known failure"
bool launchTaskHost =
    isTaskHostFactory ||
    (
        Traits.Instance.ForceAllTasksOutOfProcToTaskHost &&
        !TypeLoader.IsPartialTypeNameMatch(RegisteredName, "MSBuild") &&
        !TypeLoader.IsPartialTypeNameMatch(RegisteredName, "CallTarget"));

Problem: Only MSBuild and CallTarget are excluded, but many other tasks can use callbacks that are unsupported in TaskHost.

TaskHost's IBuildEngine Implementation Has Gaps

The OutOfProcTaskHostNode class (src/MSBuild/OutOfProcTaskHostNode.cs) implements IBuildEngine10 but has critical methods that are either:

1. Stubbed to log an error and return failure
2. No-op implementations (potentially incorrect behavior)
3. Throwing NotImplementedException

Detailed Analysis: IBuildEngine Callback Support Matrix

❌ Unsupported Callbacks (Will Cause Build Failures)

Interface	Member	Implementation in TaskHost	Impact
IBuildEngine	BuildProjectFile(projectFileName, targetNames, globalProperties, targetOutputs)	Logs MSB5022 error, returns false	Build fails
IBuildEngine2	BuildProjectFile(... toolsVersion)	Logs MSB5022 error, returns false	Build fails
IBuildEngine2	BuildProjectFilesInParallel(...)	Logs MSB5022 error, returns false	Build fails
IBuildEngine2	IsRunningMultipleNodes property	Logs MSB5022 error, returns false	Incorrect behavior + error logged
IBuildEngine3	BuildProjectFilesInParallel(... returnTargetOutputs)	Logs MSB5022 error, returns empty result	Build fails
IBuildEngine9	RequestCores(int requestedCores)	Throws NotImplementedException	Task crashes
IBuildEngine9	ReleaseCores(int coresToRelease)	Throws NotImplementedException	Task crashes

⚠️ Partially Supported (May Cause Incorrect Behavior)

Interface	Member	Implementation in TaskHost	Risk
IBuildEngine3	Yield()	Silent no-op (returns immediately)	Task may not coordinate properly with scheduler
IBuildEngine3	Reacquire()	Silent no-op (returns immediately)	Task may not coordinate properly with scheduler
IBuildEngine4	Task object registration	Local cache only - not shared with parent	Cached objects lost after task completes

✅ Fully Supported Callbacks

Interface	Member	Notes
IBuildEngine	LogErrorEvent, LogWarningEvent, LogMessageEvent, LogCustomEvent	Via LogMessagePacket
IBuildEngine	ContinueOnError, LineNumberOfTaskNode, ColumnNumberOfTaskNode, ProjectFileOfTaskNode	Passed in configuration
IBuildEngine5	LogTelemetry	Via build event
IBuildEngine6	GetGlobalProperties	From configuration
IBuildEngine7	AllowFailureWithoutError	Local property
IBuildEngine8	ShouldTreatWarningAsError	Local logic with passed-in warning sets
IBuildEngine10	EngineServices (partial)	Limited functionality

Error Message

When a task attempts to use an unsupported callback, users see:

MSB5022: The MSBuild task host does not support running tasks that perform IBuildEngine callbacks. 
If you wish to perform these operations, please run your task in the core MSBuild process instead. 
A task will automatically execute in the task host if the UsingTask has been attributed with a 
"Runtime" or "Architecture" value, or the task invocation has been attributed with an 
"MSBuildRuntime" or "MSBuildArchitecture" value, that does not match the current runtime or 
architecture of MSBuild.

This error is defined in src/Shared/Resources/Strings.shared.resx.

Code Evidence

1. Unsupported BuildProjectFile Callbacks

From src/MSBuild/OutOfProcTaskHostNode.cs lines 361-404:

/// <summary>
/// Stub implementation of IBuildEngine.BuildProjectFile.  The task host does not support IBuildEngine
/// callbacks for the purposes of building projects, so error.
/// </summary>
public bool BuildProjectFile(string projectFileName, string[] targetNames, IDictionary globalProperties, IDictionary targetOutputs)
{
    LogErrorFromResource("BuildEngineCallbacksInTaskHostUnsupported");
    return false;
}

// ... similar for all BuildProjectFile* overloads

2. Unsupported IsRunningMultipleNodes

From src/MSBuild/OutOfProcTaskHostNode.cs lines 269-277:

/// <summary>
/// Stub implementation of IBuildEngine2.IsRunningMultipleNodes.  The task host does not support this sort of
/// IBuildEngine callback, so error.
/// </summary>
public bool IsRunningMultipleNodes
{
    get
    {
        LogErrorFromResource("BuildEngineCallbacksInTaskHostUnsupported");
        return false;
    }
}

3. Unsupported Resource Management

From src/MSBuild/OutOfProcTaskHostNode.cs lines 504-516:

#region IBuildEngine9 Implementation

public int RequestCores(int requestedCores)
{
    // No resource management in OOP nodes
    throw new NotImplementedException();
}

public void ReleaseCores(int coresToRelease)
{
    // No resource management in OOP nodes
    throw new NotImplementedException();
}

#endregion

4. Silent No-op for Yield/Reacquire

From src/MSBuild/OutOfProcTaskHostNode.cs lines 409-425:

/// <summary>
/// Stub implementation of IBuildEngine3.Yield.  The task host does not support yielding, so just go ahead and silently
/// return, letting the task continue.
/// </summary>
public void Yield()
{
    return;
}

/// <summary>
/// Stub implementation of IBuildEngine3.Reacquire. The task host does not support yielding, so just go ahead and silently
/// return, letting the task continue.
/// </summary>
public void Reacquire()
{
    return;
}

Steps to Reproduce

Take Alina's prototype rebased onto main and build it.[/+]
[+]Build sdk with this version of msbuild.[/+]
[+]run the ./eng/dogfood.cmd script to dogfood it[/+]
[+]build roslyn repository via dotnet build Roslyn.sln -mt[/+]
[+]the error should manifest itself.
5. Observe build failure with MSB5022 error

Impact Assessment

Tasks Known to Use Unsupported Callbacks

Task	Callback Used	Source
MSBuild	BuildProjectFilesInParallel	Core MSBuild task
CallTarget	BuildProjectFile	Core MSBuild task
ToolTask derivatives	Yield/Reacquire (when YieldDuringToolExecution=true)	Utilities
Compiler tasks (Csc, Vbc)	Potentially various	Roslyn
RAR (ResolveAssemblyReference)	RequestCores/ReleaseCores	SDK
Custom tasks	Unknown	Third-party

Severity

• Multithreaded mode is broken for any project using tasks with unsupported callbacks
• No workaround except disabling multithreaded mode entirely
• Silent incorrect behavior for Yield/Reacquire and IsRunningMultipleNodes

Proposed Solution Direction

Approach A: Implement Callback Forwarding

Extend the TaskHost communication protocol to forward callback requests to the parent MSBuild process:

1. Add new packet types for callback requests/responses
2. Implement synchronous request-response pattern in TaskHost
3. Handle callbacks in the parent process using the real IBuildEngine

Related Files

File	Purpose
src/MSBuild/OutOfProcTaskHostNode.cs	TaskHost's IBuildEngine implementation
src/Build/Instance/TaskRegistry.cs	Task factory selection and exclusion logic
src/Build/Instance/TaskFactories/TaskHostTask.cs	Parent-side TaskHost communication
src/Build/BackEnd/Components/RequestBuilder/TaskHost.cs	In-proc TaskHost (reference implementation)
src/Framework/IBuildEngine*.cs	Interface definitions
src/Shared/TaskHostConfiguration.cs	Task configuration packet
src/Shared/TaskHostTaskComplete.cs	Task completion packet

Acceptance Criteria

1. Tasks using BuildProjectFile* callbacks work correctly in TaskHost
2. RequestCores/ReleaseCores work correctly in TaskHost
3. IsRunningMultipleNodes returns correct value in TaskHost
4. Yield/Reacquire properly coordinate with the scheduler
5. No regression in existing TaskHost functionality
6. Roslyn builds successfully with -mt mode

References

• documentation/specs/multithreading/multithreaded-msbuild.md - Multithreaded MSBuild Spec
• documentation/wiki/Nodes-Orchestration.md - Node Orchestration Documentation
• documentation/wiki/Tasks.md - Tasks Documentation