-
Notifications
You must be signed in to change notification settings - Fork 570
Add Conformance tests for the server #983
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add Conformance tests for the server #983
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR adds a new ConformanceServer sample that implements the MCP specification for testing against the MCP Conformance test suite. The server demonstrates all major MCP capabilities including tools, resources, prompts, sampling, elicitation, subscriptions, and logging. Testing shows 29 out of 30 conformance tests pass, with the single failure being a test configuration issue.
Key changes:
- Implements comprehensive tool handlers for testing different content types, logging, progress, error handling, sampling, and elicitation (including SEP-1034 and SEP-1330)
- Adds resource handlers for static text/binary content, templates with parameters, and subscribable resources
- Provides prompt handlers demonstrating simple, parameterized, embedded resource, and image-based prompts
- Includes subscription management and completion support
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| samples/ComplianceServer/ComplianceServer.csproj | Defines the project structure targeting .NET 9.0 with references to ModelContextProtocol.AspNetCore |
| samples/ComplianceServer/Program.cs | Configures the MCP server with HTTP transport, registers handlers for tools/prompts/resources, and implements subscription management |
| samples/ComplianceServer/Tools/ComplianceTools.cs | Implements 12 test tools covering text/image/audio content, logging, progress, errors, sampling, and elicitation with SEP-1034/SEP-1330 support |
| samples/ComplianceServer/Resources/ComplianceResources.cs | Provides static text/binary resources and a parameterized template resource for conformance testing |
| samples/ComplianceServer/Prompts/CompliancePrompts.cs | Implements simple, parameterized, embedded resource, and image-based prompts |
| samples/ComplianceServer/appsettings.json | Standard ASP.NET Core configuration for logging and allowed hosts |
| samples/ComplianceServer/appsettings.Development.json | Development-specific logging configuration |
| samples/ComplianceServer/Properties/launchSettings.json | Defines HTTP and HTTPS launch profiles for localhost:3001 |
| samples/ComplianceServer/ComplianceServer.http | Provides sample HTTP requests for testing the server endpoints |
| ModelContextProtocol.slnx | Adds the ComplianceServer project to the solution |
Comments suppressed due to low confidence (1)
samples/ComplianceServer/Tools/ComplianceTools.cs:1
- Field names in result content access don't match the schema property names defined above. The code accesses 'stringField', 'numberField', and 'booleanField', but the schema defines 'name', 'age', 'score', 'status', and 'verified'.
using ModelContextProtocol;
tests/ModelContextProtocol.ConformanceServer/ModelContextProtocol.ConformanceServer.csproj
Show resolved
Hide resolved
| public static IEnumerable<PromptMessage> PromptWithEmbeddedResource( | ||
| [Description("URI of the resource to embed")] string resourceUri) | ||
| { | ||
| return [ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similarly here, you could return a PromptMessage instead of an enumerable of prompt messages, if you want.
stephentoub
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Neat
e928861 to
550cdc4
Compare
|
@stephentoub I think I've addressed all your comments. Copilot wrote the test class -- I'm not sure it is right so please give that a close look. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 15 comments.
tests/ModelContextProtocol.ConformanceTests/ConformanceTests.cs
Outdated
Show resolved
Hide resolved
tests/ModelContextProtocol.ConformanceTests/ConformanceTests.cs
Outdated
Show resolved
Hide resolved
tests/ModelContextProtocol.ConformanceTests/ConformanceTests.cs
Outdated
Show resolved
Hide resolved
550cdc4 to
ed0adcf
Compare
Motivation and Context
This PR adds a new sample server for use with the [MCP Conformance test suite].
How Has This Been Tested?
Tested with the server conformance test suite and all pass except one which I believe is a test error.
The failing scenario is tools-call-with-logging, which fails because it did not set the logging level at the start of the test.
Breaking Changes
No breaking changes.
Types of changes
Checklist
Additional context