Parse GenAI tool definitions from span attributes and display in UI with accordion layout #12863
Conversation
|
@copilot Add new overview tab to GenAIVisualizerDialog. Tab should be:
|
Done in f2628f7. Added "Tools" tab between "Input & output" and "Details" tabs with: Tab features:
Tab content displays:
Note: Since this is a UI change requiring runtime data, I cannot provide a screenshot in this environment. The tab will be visible in the GenAIVisualizerDialog when tool definitions are present in the span attributes. |
|
🚀 Dogfood this PR with:
curl -fsSL https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.sh | bash -s -- 12863Or
iex "& { $(irm https://raw.githubusercontent.com/dotnet/aspire/main/eng/scripts/get-aspire-cli-pr.ps1) } 12863" |
|
@copilot Change tools tab to display a list of tools in a FluentAccodion control. Each tool is a FluentAccordionItem, with the type, name and description in the heading. The type should be displayed in a badge, have a fixed width, and have a color depending on the type. Right now the only known types are "function" and "mcp". Inside the tab display a table of parameters, with the name, type and description. See |
Done in 9e20e6b. Updated Tools tab to use FluentAccordion with improved layout: FluentAccordion features:
Parameters table:
The layout now matches the Swagger UI style from the reference image, with collapsible accordion items and a clean parameter table. |
JamesNK
force-pushed
the
copilot/parse-genai-tool-definitions
branch
from
9e20e6b to
1e5809b
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR implements parsing and display of GenAI tool definitions from OpenTelemetry span attributes according to the gen_ai.tool.definitions semantic convention. The feature adds a new Tools tab to the GenAI visualizer dialog that shows tool definitions in an accordion layout with parameter details.
Key changes:
- Added
ToolDefinitionmodel class leveragingOpenApiSchemafrom Microsoft.OpenApi v1.6.25 for parameter structure - Implemented custom JSON parsing with error isolation that converts JSON to
OpenApiSchemaobjects - Added Tools tab UI with accordion display showing tool name, description, parameters table, and required field indicators
Reviewed Changes
Copilot reviewed 28 out of 29 changed files in this pull request and generated 9 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Aspire.Dashboard/Model/GenAI/GenAIMessages.cs | Added ToolDefinition model class with Type, Name, Description, and Parameters properties |
| src/Aspire.Dashboard/Model/GenAI/ToolDefinitionViewModel.cs | Created view model wrapper for tool definitions with UI state (Expanded property) |
| src/Aspire.Dashboard/Model/GenAI/GenAIVisualizerDialogViewModel.cs | Added ToolDefinitions property, parsing logic for gen_ai.tool.definitions attribute, and ParseOpenApiSchema helper method |
| src/Aspire.Dashboard/Model/GenAI/GenAIHelpers.cs | Added GenAIToolDefinitions constant for the attribute key |
| src/Aspire.Dashboard/Model/GenAI/GenAIItemViewModel.cs | Added support for tool response badge category |
| src/Aspire.Dashboard/Model/GenAI/GenAIItemPartViewModel.cs | Improved tool call request display to handle empty arguments |
| src/Aspire.Dashboard/Components/Dialogs/GenAIVisualizerDialog.razor | Added Tools tab with accordion UI, tool definition/call navigation buttons, and parameters table |
| src/Aspire.Dashboard/Components/Dialogs/GenAIVisualizerDialog.razor.cs | Added ViewToolDefinition and TryGetToolCall helper methods, icon constants |
| src/Aspire.Dashboard/Components/Dialogs/GenAIVisualizerDialog.razor.css | Added CSS for tools list, accordion styling, parameters table, and tool response category colors |
| src/Aspire.Dashboard/wwwroot/css/app.css | Added CSS variables for property grid and tool response colors in light/dark themes |
| src/Aspire.Dashboard/Resources/Dialogs.resx | Added localization strings GenAIToolsTabText, GenAINoTools, and GenAIMessageCategoryToolResponse |
| src/Aspire.Dashboard/Resources/Dialogs.Designer.cs | Generated code for new localization resources |
| src/Aspire.Dashboard/Resources/xlf/*.xlf | Updated all localization files with new strings marked for translation |
| src/Aspire.Dashboard/Aspire.Dashboard.csproj | Added Microsoft.OpenApi package reference |
| Directory.Packages.props | Added Microsoft.OpenApi version 1.6.25 package version |
| tests/Aspire.Dashboard.Tests/Model/GenAIVisualizerDialogViewModelTests.cs | Added comprehensive tests for tool definition parsing, invalid JSON handling, and missing definitions |
| playground/GitHubModelsEndToEnd/GitHubModelsEndToEnd.WebStory/Components/Pages/Home.razor | Added example tool definitions (GeneratePlotTwist) to demonstrate the feature |
Files not reviewed (1)
- src/Aspire.Dashboard/Resources/Dialogs.Designer.cs: Language not supported
|
cc @stephentoub. Using the new tool definitions telemetry from sem conventions and MEAI. Looks like only functions have useful information so UI is limited to showing just them |
|
@copilot Move hard coded text in Aspire.Dashboard to resource files. |
Moved hardcoded strings in GenAI Tools tab to localized resource strings: - "Name" → GenAIToolParameterName - "Type" → GenAIToolParameterType - "Description" → GenAIToolParameterDescription - "Required parameter" → GenAIToolRequiredParameter - "No parameters" → GenAIToolNoParameters All strings are now properly localized and can be translated. Co-authored-by: JamesNK <[email protected]>
Done in ca58cbe. Moved all hardcoded text in the Tools tab to resource files: New resource strings:
All UI text in the GenAI Tools tab is now properly localized and can be translated to all supported languages. |
mitchdenny
left a comment
mitchdenny
left a comment
There was a problem hiding this comment.
Code looks good. I think you might want to iterate a bit on the UX but won't block this PR since its strictly better functionality than what we have today.
I found that it took me a moment when looking at this UX to intuit what happened when I clocked on Tool call and Tool definition. I wonder if it would be better to show them inline somehow rather than either jumping to the root of the tree or back up to the previous tool call.
I can see why it was done this way for implementation expediency though.
Parse GenAI Tool Definitions Implementation
gen_ai.tool.definitionsattribute in GenAIHelpers.csImplementation Details
Tool Definition Models:
ToolDefinitionclass withType,Name,Description, andParameterspropertiesParametersproperty usesOpenApiSchemafrom Microsoft.OpenApi v1.6.25OpenApiSchemaobjectsTools Tab UI:
FluentAccordioncontrol with each tool as aFluentAccordionItemLocalized Strings:
All tests passing (14/14 GenAI tests).
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.