Agent callbacks

[tkattkat]

Summary

Adds callback support to the Stagehand agent for both streaming and non-streaming execution modes, allowing users to hook into various stages of agent execution.

Changes

New Types (lib/v3/types/public/agent.ts)

Added callback interfaces for agent execution:

• AgentCallbacks - Base callbacks shared between modes:

  • prepareStep - Modify settings before each LLM step
  • onStepFinish - Called after each step completes

• AgentExecuteCallbacks - Non-streaming mode callbacks (extends AgentCallbacks)

• AgentStreamCallbacks - Streaming mode callbacks (extends AgentCallbacks):

  • onChunk - Called for each streamed chunk
  • onFinish - Called when stream completes
  • onError - Called on stream errors
  • onAbort - Called when stream is aborted

• AgentExecuteOptionsBase - Base options without callbacks

• AgentExecuteOptions - Non-streaming options with AgentExecuteCallbacks

• AgentStreamExecuteOptions - Streaming options with AgentStreamCallbacks

Handler Updates (lib/v3/handlers/v3AgentHandler.ts)

• Modified createStepHandler to accept optional user callback
• Updated execute() to pass callbacks to generateText
• Updated stream() to pass callbacks to streamText

Type Safety

Added compile-time enforcement that streaming-only callbacks (onChunk, onFinish, onError, onAbort) can only be used with stream: true:

// ✅ Works - streaming callbacks with stream: true
const agent = stagehand.agent({ stream: true });
await agent.execute({
  instruction: "...",
  callbacks: { onChunk: async (chunk) => console.log(chunk) }
});

// ❌ Compile error - streaming callbacks without stream: true
const agent = stagehand.agent({ stream: false });
await agent.execute({
  instruction: "...",
  callbacks: { onChunk: async (chunk) => console.log(chunk) }
  // Error: "❌ This callback requires 'stream: true' in AgentConfig..."
});

Type Castings Explained

Several type assertions were necessary due to TypeScript's limitations with conditional types:

1. Callback extraction in handlers

const callbacks = (instructionOrOptions as AgentExecuteOptions).callbacks as
  | AgentExecuteCallbacks
  | undefined;

Why: instructionOrOptions can be string | AgentExecuteOptions. When it's a string, there are no callbacks. We cast after the prepareAgent call because at that point we know it's been resolved to options.

2. Streaming vs non-streaming branch in v3.ts

result = await handler.execute(
  instructionOrOptions as string | AgentExecuteOptions,
);

Why: The implementation signature accepts string | AgentExecuteOptions | AgentStreamExecuteOptions to satisfy both overloads, but within the non-streaming branch we know it's the non-streaming type. TypeScript can't narrow based on the isStreaming runtime check.

3. Error fallback in stream()

return {
  textStream: (async function* () {})(),
  result: resultPromise,
} as unknown as AgentStreamResult;

Why: When prepareAgent fails in streaming mode, we return a minimal object with just textStream and result. This doesn't satisfy all properties of StreamTextResult, but the result promise will reject with the actual error. The double cast (as unknown as) is needed because TypeScript knows this partial object doesn't match the full type.

Usage Example

const agent = stagehand.agent({
  stream: true,
  model: "anthropic/claude-sonnet-4-20250514",
});

const result = await agent.execute({
  instruction: "Search for something",
  maxSteps: 20,
  callbacks: {
    prepareStep: async (ctx) => {
      console.log("Preparing step...");
      return ctx;
    },
    onStepFinish: async (event) => {
      console.log(`Step finished: ${event.finishReason}`);
      if (event.toolCalls) {
        for (const tc of event.toolCalls) {
          console.log(`Tool used: ${tc.toolName}`);
        }
      }
    },
    onChunk: async (chunk) => {
      // Process each chunk
    },
    onFinish: (event) => {
      console.log(`Completed in ${event.steps.length} steps`);
    },
  },
});

for await (const chunk of result.textStream) {
  process.stdout.write(chunk);
}

const finalResult = await result.result;
console.log(finalResult.message);

Testing

• Added agent-callbacks.spec.ts with tests for:
  • Non-streaming callbacks (onStepFinish, prepareStep)
  • Streaming callbacks (onChunk, onFinish, prepareStep, onStepFinish)
  • Combined callback usage
  • Tool call information in callbacks

---

Summary by cubic

Adds lifecycle callbacks to the Stagehand agent for both non-streaming and streaming modes, so users can hook into steps, chunks, finish, and errors. Strong type safety prevents using streaming-only callbacks without stream: true.

Why:

• Give users fine-grained control over agent execution.
• Prevent misuse of streaming callbacks with clear compile-time errors.

What:

• Types
  • Added AgentCallbacks, AgentExecuteCallbacks, AgentStreamCallbacks, AgentExecuteOptionsBase, AgentStreamExecuteOptions.
  • Compile-time checks restrict onChunk/onFinish/onError/onAbort to stream: true.
• Handlers
  • createStepHandler now forwards user onStepFinish.
  • execute() and stream() pass prepareStep/onStepFinish and streaming callbacks to generateText/streamText.
• V3 and Cache
  • V3 agent() overloads accept both execute and stream option shapes; routing updated accordingly.
  • AgentCache.sanitizeExecuteOptions now uses AgentExecuteOptionsBase.
  • Callbacks require experimental: true; agent() throws if callbacks are provided without experimental.
• Tests
  • Added agent-callbacks.spec.ts covering non-streaming, streaming, and combined callbacks, including tool call data.

Test Plan:

• Automated: playwright tests in packages/core/lib/v3/tests/agent-callbacks.spec.ts verify:
  • Non-streaming: prepareStep, onStepFinish, tool call info.
  • Streaming: onChunk, onFinish, prepareStep, onStepFinish.
  • Combined usage with multiple callbacks.
• Type checks: public API type tests ensure new option and callback shapes are exported and correct.

^{Written for commit bc1d2a6. Summary will update automatically on new commits.}

[changeset-bot]

🦋 Changeset detected

Latest commit: bc1d2a6

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/stagehand-evals	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[greptile-apps]

Greptile Overview

Greptile Summary

Adds lifecycle callback support to Stagehand agents, enabling users to hook into various stages of agent execution for both streaming and non-streaming modes.

Key Changes:

• New callback interfaces: AgentCallbacks (shared), AgentExecuteCallbacks (non-streaming), and AgentStreamCallbacks (streaming)
• Compile-time type safety preventing streaming callbacks (onChunk, onFinish, onError, onAbort) without stream: true
• Handler updates to extract and forward callbacks to AI SDK's generateText and streamText
• Callback exclusion from cache keys via AgentExecuteOptionsBase
• Comprehensive test coverage for all callback scenarios

Implementation Quality:

• Clean separation of concerns with callbacks properly typed and routed
• Type assertions are justified and documented in PR description
• Cache integration correctly excludes callbacks from cache keys
• Tests verify callback execution, tool call information capture, and multi-callback usage

Confidence Score: 5/5

• This PR is safe to merge with minimal risk
• The implementation is well-architected with strong type safety, comprehensive test coverage, and proper separation of concerns. The callback system correctly integrates with the AI SDK, and the cache exclusion logic prevents callback pollution in cache keys. Type assertions are necessary due to TypeScript limitations with conditional types and are well-documented.
• No files require special attention

Important Files Changed

File Analysis

Filename	Score	Overview
packages/core/lib/v3/types/public/agent.ts	5/5	Adds comprehensive callback type definitions with compile-time safety guards preventing streaming callbacks in non-streaming mode
packages/core/lib/v3/handlers/v3AgentHandler.ts	5/5	Updates handler to forward user callbacks to AI SDK, properly extracting and passing callbacks to generateText and streamText
packages/core/lib/v3/v3.ts	4/5	Updates agent() overloads and routing logic to support callback parameters in both streaming and non-streaming execution paths

Sequence Diagram

sequenceDiagram
    participant User
    participant V3
    participant V3AgentHandler
    participant LLMClient
    participant AISDKCallbacks

    User->>V3: agent({ stream: true/false })
    V3-->>User: AgentInstance

    User->>V3: agent.execute({ instruction, callbacks })
    V3->>V3: prepareAgentExecution()
    V3->>V3AgentHandler: new V3AgentHandler()
    
    alt Non-streaming (stream: false)
        V3->>V3AgentHandler: execute(instructionOrOptions)
        V3AgentHandler->>V3AgentHandler: prepareAgent()
        V3AgentHandler->>V3AgentHandler: extract callbacks from options
        V3AgentHandler->>LLMClient: generateText({ prepareStep, onStepFinish })
        loop Each step
            LLMClient->>AISDKCallbacks: prepareStep(context)
            AISDKCallbacks-->>LLMClient: modified context
            LLMClient->>LLMClient: execute step
            LLMClient->>AISDKCallbacks: onStepFinish(event)
            AISDKCallbacks->>V3AgentHandler: createStepHandler(event)
            V3AgentHandler->>User: callbacks.onStepFinish(event)
        end
        V3AgentHandler-->>V3: AgentResult
        V3-->>User: AgentResult
    else Streaming (stream: true)
        V3->>V3AgentHandler: stream(instructionOrOptions)
        V3AgentHandler->>V3AgentHandler: prepareAgent()
        V3AgentHandler->>V3AgentHandler: extract callbacks from options
        V3AgentHandler->>LLMClient: streamText({ prepareStep, onStepFinish, onChunk, onFinish, onError })
        loop Each step
            LLMClient->>AISDKCallbacks: prepareStep(context)
            AISDKCallbacks-->>LLMClient: modified context
            loop Each chunk
                LLMClient->>User: callbacks.onChunk(chunk)
            end
            LLMClient->>AISDKCallbacks: onStepFinish(event)
            AISDKCallbacks->>V3AgentHandler: createStepHandler(event)
            V3AgentHandler->>User: callbacks.onStepFinish(event)
        end
        LLMClient->>User: callbacks.onFinish(event)
        V3AgentHandler-->>V3: AgentStreamResult
        V3-->>User: AgentStreamResult
        User->>User: iterate textStream
        User->>User: await result promise
    end

Loading

[greptile-apps]

_{6 files reviewed, no comments}

_{Edit Code Review Agent Settings | Greptile}

[cubic-dev-ai]

No issues found across 6 files

[cubic-dev-ai]

No issues found across 7 files