Skip to content

Commit cfecb67

Browse files
ohltylerkolchfa-awsnatebower
authored
Update agentic search UI for 3.4 (#11584)
* Update agentic search UI for 3.4 Signed-off-by: Tyler Ohlsen <[email protected]> * cleanup Signed-off-by: Tyler Ohlsen <[email protected]> * update agent link Signed-off-by: Tyler Ohlsen <[email protected]> * more details on responsefilter callout Signed-off-by: Tyler Ohlsen <[email protected]> * more cleanup, add images Signed-off-by: Tyler Ohlsen <[email protected]> * update sentece Signed-off-by: Tyler Ohlsen <[email protected]> * comments Signed-off-by: Tyler Ohlsen <[email protected]> * Doc review Signed-off-by: Fanit Kolchina <[email protected]> * Update screenshots Signed-off-by: Tyler Ohlsen <[email protected]> * Apply suggestions from code review Signed-off-by: Nathan Bower <[email protected]> --------- Signed-off-by: Tyler Ohlsen <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: Nathan Bower <[email protected]> Co-authored-by: Fanit Kolchina <[email protected]> Co-authored-by: Nathan Bower <[email protected]>
1 parent 84ff134 commit cfecb67

16 files changed

+128
-20
lines changed

_vector-search/ai-search/building-agentic-search-flows.md

Lines changed: 128 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -10,42 +10,150 @@ nav_order: 20
1010
**Introduced 3.3**
1111
{: .label .label-purple }
1212

13-
This is an experimental UI feature. For updates on the progress of the feature or if you want to leave feedback, join the discussion on the [OpenSearch forum](https://forum.opensearch.org/).
13+
This is an experimental UI feature. For updates on the progress of the feature or if you want to leave feedback, join the discussion on the [OpenSearch forum](https://forum.opensearch.org/).
1414
{: .warning}
1515

16-
[Agentic search]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/) lets you ask questions in natural language and have OpenSearch agents plan and execute the retrieval automatically. OpenSearch Dashboards offers an intuitive UI for configuring agents, equipping agents with different tools, and executing agentic searches.
16+
[Agentic search]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/) allows you to ask questions in natural language and have OpenSearch agents plan and execute retrieval automatically. This feature combines large language models (LLMs) with OpenSearch's search capabilities to provide intelligent, context-aware search experiences.
17+
18+
OpenSearch Dashboards provides an intuitive interface for configuring agents, equipping agents with different tools, executing agentic searches, and integrating agentic search into your applications.
19+
20+
The following image shows the agentic search workflow interface with agent configuration options on the left and search execution capabilities on the right.
21+
22+
![Agentic search editor]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-editor.png)
1723

1824
## Prerequisites
1925

2026
Before configuring agentic search, ensure that you fulfill the following prerequisites.
2127

2228
### Provision ML resources
2329

24-
To configure new agents, be sure to first provision appropriate models. For working examples, see [Model configuration]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/agent-customization/#model-configuration).
30+
To configure new agents, first provision appropriate models. For working examples, see [Model configuration]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/agent-customization/#model-configuration).
2531

2632
### Ingest data
2733

28-
Ensure that you have a sufficient number of documents in your cluster to reasonably evaluate your agentic searches. For more information, see [Agentic search]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/).
34+
Ensure that you have a sufficient number of documents in your cluster to reasonably evaluate your agentic searches.
35+
36+
## Accessing the plugin
37+
38+
To access the plugin, go to **OpenSearch Dashboards** and select **OpenSearch Plugins** > **AI Search Flows** from the top menu.
39+
40+
## Configuring agents
41+
42+
Agents are highly customizable and can be configured in multiple ways depending on your use case. The following image shows a fully configured conversational agent.
43+
44+
![Agent configuration]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agent-configuration.png)
45+
46+
### Agent types
47+
48+
[Flow agents]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/agents/flow/) are optimized for speed and simplicity in query generation. [Conversational agents]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/agents/conversational/) can be configured with more tools and functionality for deeper reasoning. Agent types can only be configured **once** during agent creation.
49+
50+
When configuring flow agents, some models require you to manually add a suitable `response_filter` in the **Query Planning** tool. For more information, see [Register a flow agent]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/flow-agent/#step-4-register-a-flow-agent). Supported providers are OpenAI and Amazon Bedrock Converse.
51+
{: .note}
52+
53+
### Models
54+
55+
In conversational agents, models are responsible for intelligent reasoning, including tool orchestration, connecting to external sources, and generating appropriate query domain-specific language (DSL) queries. In flow agents, tools run sequentially, and models in the **Query Planning** tool are used only for query generation. Different models are optimized for different scenarios: cost-efficient, fast inference versus resource-intensive, deep-reasoning approaches.
56+
57+
For a list of suggested models compatible with agentic search, see [Model configuration]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/agent-customization/#model-configuration).
58+
59+
### Tools
60+
61+
All agents require the **Query Planning** tool to execute agentic searches. Query generation can be entirely LLM-generated (default) or guided by predefined search templates. Search templates help maintain control over generated queries by directing the model to use known, tested, and performant query patterns.
62+
63+
The following image shows the search templates configuration interface within the **Query Planning** tool.
64+
65+
![Search templates]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-search-template.png)
66+
67+
Other prebuilt tools are available for conversational agents, including **Search Index**, **List Index**, **Index Mapping**, and **Web Search**. Enable these to extend your agent's capabilities. While these represent the most commonly used tools in agentic search, several additional [OpenSearch tools are available]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/index/) and can be configured manually in the **JSON** view.
68+
69+
### MCP servers
70+
71+
To allow conversational agents to access more external tools, integrate them with Model Context Protocol (MCP) servers. To limit the tools that the agents can access, configure filters for each server under **Tool filters**. For more information, see [Using external MCP servers]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/mcp-server).
72+
73+
## Running agentic searches
74+
75+
Test how agents perform with different indexes and search queries. Collapse the **Configure agent** panel to focus on executing searches and analyzing results. The following image shows a search for `mens blue shirts` against the `demo_amazon_fashion` index, including relevant result images.
76+
77+
![Agentic search configuration]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-configuration.png)
78+
79+
### Index
80+
81+
Try different indexes in your cluster. To view index details, select the **Inspect** button. For conversational agents, you can select **All indexes** and let the agent choose the appropriate index.
82+
83+
### Agent
84+
85+
Try different agents you've created. To view agent details, select the **Inspect** button.
86+
87+
### Query
88+
89+
Test different natural language queries. To specify fields for the agent to focus on in your selected index, select **Add query fields**. To edit the full `agentic` search query directly, toggle to the **JSON** view. For conversational agents, select **Continue conversation** after a search to maintain context for future searches. To discard conversational history and start again, select **Clear conversation**.
90+
91+
### Running searches
92+
93+
To run an agentic search, select **Search**. The process may take several seconds while the agent reasons about the query, analyzes index mappings, and performs tool orchestration. If the search takes too long or you want to try a different search, select **Stop**.
94+
95+
After the search completes, view the results in the following sections:
96+
97+
- **Generated query**: The query DSL that the agent generated and executed against your cluster
98+
- **Search results**: The search response with available tabs based on the results:
99+
- **Aggregations**: Displayed when the response contains aggregations
100+
- **Visual hits**: Displayed when document hits contain images
101+
- **Hits**: Displayed when the response contains any hits
102+
- **Raw response**: Always available for detailed inspection
103+
104+
For conversational agents, select **View agent summary** to see a step-by-step breakdown of the agent's actions, including the sequence of tools used and the reasoning behind each step.
105+
106+
The following image shows the search results interface with different viewing tabs and the agent summary option.
107+
108+
![Agentic search results]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-example-search-results.png)
109+
110+
### Using agentic search in your application
111+
112+
Select **Export** to view all of the underlying resources needed for using agentic search in your downstream application, including the agent and search pipeline details.
113+
114+
The following image shows the export dialog with code examples for integrating agentic search into your applications.
115+
116+
![Agentic search export]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-export.png)
29117

30118
## Example: Product search with GPT-5
31119

32-
This example uses a deployed OpenAI GPT-5 model described in [this documentation]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/agent-customization/#gpt-5-recommended).
120+
This example uses a deployed OpenAI [GPT-5 model]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/agent-customization/#gpt-5-recommended) and an index generated from the [fashion product images dataset](https://www.kaggle.com/datasets/paramaggarwal/fashion-product-images-dataset).
33121
{: .note}
34122

35-
To build and test your agentic search workflow in OpenSearch Dashboards, follow these steps:
123+
1. Navigate to the **AI Search Flows** plugin. On the **Workflows** page, select the **New workflow** tab, as shown in the following image. In the **Agentic Search** template, select **Create**.
36124

37-
1. Go to **OpenSearch Dashboards** and select **OpenSearch Plugins** > **AI Search Flows** from the top menu.
38-
1. On the **Workflows** page, select the **New workflow** tab, as shown in the following image. In the **Agentic Search** template, select **Create**.
39125
![New workflow page]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/new-workflow-page.png)
40-
1. Provide a unique workflow **Name** and an optional **Description**, as shown in the following image. Then select **Create** to create your workflow. You are automatically directed to the workflow editor, where you can begin configuring the agent.
41-
![Quick configure modal]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-quick-configure-modal.png)
42-
1. Under **Configure agent**, select **Create new agent**, as shown in the following image. Once you configure an agent, you can select or update existing agents. For full agent customization, toggle to the `JSON` view and edit directly.
43-
![Agentic search workflow editor]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-editor.png)
44-
1. Under **Agent**, enter a unique **Name** and **Description** for the agent, as shown in the following image. Under **Tools** > **Query Planning** > **Query planning model**, select **OpenAI GPT-5**. Then select **Create agent**.
45-
![Agent configuration]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agent-configuration.png)
46-
1. Under **Test flow** > **Index**, select the index you'd like to search, as shown in the following image. Under **Test flow** > **Query**, enter a natural language query. Optionally, specify query fields for the agent to search in your selected index. To edit the full `agentic` search query directly, toggle to the `JSON` view. Then select **Search**. The agent may take several seconds to execute.
47-
![Agent searching]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agent-searching.png)
48-
1. Under **Generated query**, view the query domain-specific language (DSL) that the agent generated and ran against your cluster.
49-
![Agent query]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-agent-query.png)
50-
1. Under **Search results**, select **Raw response**, the formatted table of **Hits**, or **Aggregations** to view the results.
51-
![Search results]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-results.png)
126+
2. Provide a unique workflow **Name** and an optional **Description**, as shown in the following image. Then select **Create** to create your workflow. You are automatically directed to the workflow editor, where you can begin configuring the agent.
127+
128+
![Quick configure modal]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-quick-configure-modal.png)
129+
3. Under **Configure agent**, select **Create new agent**. Enter a unique **Name**, optionally provide a **Description**, and under **Model**, select the deployed **OpenAI GPT-5** model. Finally, select **Create agent** at the bottom.
130+
131+
![Agentic search configuration]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-example-configuration.png)
132+
4. Under **Agentic search**, select the index you'd like to search against, or leave the default as **All indices** and let the agent decide. The agent will already be selected.
133+
5. Under **Query**, enter a natural language query about your data, as shown in the following image. Select **Search** to run an agentic search.
134+
135+
![Agentic search]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-example-search.png)
136+
6. View the agent's generated query, the search hits, and agent summary.
137+
138+
The following image shows the generated query DSL that the agent created for the search.
139+
140+
![Agentic search query]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-example-query.png)
141+
142+
The following image shows the search results with product images and details.
143+
144+
![Agentic search hits]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-example-hits.png)
145+
146+
The following image shows the agent's step-by-step reasoning and tool usage summary.
147+
148+
![Agentic search summary]({{site.url}}{{site.baseurl}}/images/dashboards-flow-framework/agentic-search-example-summary.png)
149+
150+
7. Optionally, tune your agent by trying different tools, models, and MCP server integrations to see how the agent performs on different queries against your data.
151+
152+
## Next steps
153+
154+
- Try agentic search on the [ML Playground](https://ml.playground.opensearch.org/app/opensearch-flow#/workflows/WAmxgJoBWVNV3bhKKnGx?configureAgent=false).
155+
- Learn more about [agentic search]({{site.url}}{{site.baseurl}}/vector-search/ai-search/agentic-search/).
156+
- Read [this blog post on agentic search](https://opensearch.org/blog/introducing-agentic-search-in-opensearch-transforming-data-interaction-through-natural-language/).
157+
- Explore [OpenSearch agents]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/agents/index/).
158+
- Join the discussion on the [OpenSearch forum](https://forum.opensearch.org/t/use-cases-and-general-feedback-for-agentic-search/27488).
159+
- Report issues on [GitHub](https://github.com/opensearch-project/dashboards-flow-framework).
3.19 KB
Loading
-38.6 KB
Binary file not shown.
-106 KB
Binary file not shown.
200 KB
Loading
150 KB
Loading
112 KB
Loading
265 KB
Loading
116 KB
Loading
151 KB
Loading

0 commit comments

Comments
 (0)