data: string in an input_image message item does not work with some providers

[raventos]

Bug for @openai/agents-extensions
Version: Introduced after 0.1.0 (works in 0.1.0, broken in newer versions)

Description:
When using input_image with a URL (e.g., Gemini Files API URI), the image fails to process with the error:
AI_APICallError: Invalid value at 'contents[0].parts[0].inline_data.data' (TYPE_BYTES), Base64 decoding failed for "https://generativelanguage.googleapis.com/v1beta/files/..."

Root Cause:
In aiSdk.js, the input_image handler changed from:

// v0.1.0 (correct)
data: url,  // passes URL object

to:

// newer version (broken)
data: url.toString(),  // converts to string

This breaks @ai-sdk/google which checks part.data instanceof URL to decide between fileData.fileUri (correct) vs inlineData.data (incorrect for URLs).

Expected: Gemini API receives fileData: { fileUri: "https://..." }
Actual: Gemini API receives inlineData: { data: "https://..." } → base64 decode fails
Workaround: Pin to @openai/agents-extensions@0.1.0

[seratch]

Thanks for reporting this issue. If you're using the latest version (0.3.x) of the extension module, it relies on ai-sdk's newer major version and the current code should be compatible with the requirements by ai-sdk (in terms of TS typing). If you need to use the older version of the ai-sdk adapter, you can fork the v1 compatible module at https://github.com/openai/openai-agents-js/tree/main/examples/ai-sdk-v1 and use it for your use case.

If you haven't tried v0.3.x yet, please try the latest version out.

[raventos]

Thanks. I am however on the latest "@ai-sdk/google": "^2.0.44" and the issue persists.
Only workaround is to downgrade to an older @openai/agents-extensions (0.1.0 seems to work).

[seratch]

Thanks for your reply. I've checked the code and found that now the ai-sdk accepts both string and URL for the property. So, your suggestion should work in the latest version. I will come up with a pull request fixing it.