QSE-4732 alpha release testing support #72
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ankurgitks
commented
Nov 13, 2025
ankurgitks
commented
- Removed redundant azure-ai-validation directory (21 files, 6,771 lines).
- Replaced hardcoded secrets with placeholders in shell scripts.
- Consolidated all testing into alpha-release-testing framework with 5 test apps (LangChain, LangGraph, Traceloop, Direct * Azure, Retail Shop),
- comprehensive documentation,
- Docker/Kubernetes deployment configs, and
- automated test runners
|
All contributors have signed the CLA ✍️ ✅ |
Contributor
|
@ankurgitks Looks like the lint is not getting triggere. Please make sure it passes lint. |
ankurgitks
force-pushed
the
QSE-4732-clean
branch
from
bf13176 to
22a4e8e
Compare
Author
|
I have read the CLA Document and I hereby sign the CLA |
…ability This PR introduces a complete testing framework for validating AI observability features in Splunk O11y, including multiple instrumentation patterns and evaluation metrics testing. Key Changes: - Add alpha-release-testing framework with 5 test applications - Add retail_shop_langchain_app.py: 3-agent retail system with unified traces - Add direct_azure_openai_app.py: Manual GenAI instrumentation without frameworks - Add comprehensive documentation for zero-code vs manual instrumentation - Add Docker and Kubernetes deployment configurations - Add environment variables reference (30+ configuration options) - Fix LangchainInstrumentor entry point case sensitivity issue - Remove redundant azure-ai-validation directory - Replace all hardcoded API keys and tokens with placeholders - Update pyproject.toml to exclude examples from linting - Update requirements to use PyPI version of traceloop translator Applications: 1. retail_shop_langchain_app.py - LangChain auto-instrumentation 2. langchain_evaluation_app.py - Deterministic evaluation testing 3. langgraph_travel_planner_app.py - Complex workflow orchestration 4. direct_azure_openai_app.py - Manual GenAI instrumentation 5. traceloop_travel_planner_app.py - Framework-agnostic instrumentation Validation: - All apps tested on RC0 environment - Evaluation metrics visible on all agents - Unified trace structure confirmed - Token usage metrics accurate - Documentation matches customer requirements Fixes: QSE-4732 Signed-off-by: ankurs <[email protected]>
ankurgitks
force-pushed
the
QSE-4732-clean
branch
from
22a4e8e to
464d9a3
Compare
srv-gh-o11y-gdi-cla
added a commit
to splunk/cla-agreement
that referenced
this pull request
Nov 21, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.