Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 16 additions & 13 deletions backend/app/classification/prompt.py
Original file line number Diff line number Diff line change
@@ -1,26 +1,29 @@
DEVREL_TRIAGE_PROMPT = """Analyze this message to determine if it needs DevRel assistance.
DEVREL_TRIAGE_PROMPT = """
Analyze this message to determine if it needs DevRel assistance.
Message: {message}
Context: {context}
DevRel handles:
- Technical questions about projects/APIs
- Developer onboarding and support
- Bug reports and feature requests
- Community discussions about development
- Documentation requests
- General developer experience questions
DevRel should be triggered ONLY if:
- The user explicitly mentions or tags the DevRel AI bot (e.g., "@Devr.AI", "@devrel", etc.)
- The message is a direct question about setting up the project, contributing, build/runtime errors, or anything clearly about this repository’s development.
- The user asks about documentation, onboarding, or GitHub issues/PRs in this repo.
DO NOT trigger DevRel for:
- General conversation, greetings, or unrelated chat
- Messages between users that do NOT mention the bot
Respond ONLY with JSON:
{{
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not very sure how this will affect the model response, as the model can also respond with JSON as markdown, I faced this a lot while interacting with the system. Could you please check a bit on this part? A few interactions would be awesome.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for highlighting this—it's a really important point! I've noticed as well that sometimes the model returns JSON wrapped in Markdown code blocks, which can trip up our downstream processing if we're expecting plain JSON.

To make sure the new prompt works as intended, I'll check a variety of typical user inputs—including ones where the bot is mentioned directly, general greetings, technical questions, and even some off-topic messages. I'll pay close attention to whether the model outputs just raw JSON or still adds Markdown formatting. If I see the model consistently adding code blocks, we can tweak the prompt to explicitly ask for plain JSON only.

If you have any particular examples where this was an issue, let me know and I'll make sure to include them in the checks. Otherwise, I'll report back soon with what I find and suggest any further changes if needed!

"needs_devrel": true/false,
"priority": "high|medium|low",
"reasoning": "brief explanation"
}}
}
Examples:
- "How do I contribute?" → {{"needs_devrel": true, "priority": "high", "reasoning": "Onboarding question"}}
- "What's for lunch?" → {{"needs_devrel": false, "priority": "low", "reasoning": "Not development related"}}
- "API is throwing errors" → {{"needs_devrel": true, "priority": "high", "reasoning": "Technical support needed"}}
- "@Devr.AI how do I build this?" → {"needs_devrel": true, "priority": "high", "reasoning": "Explicitly tagged bot for setup help"}
- "Hi everyone!" → {"needs_devrel": false, "priority": "low", "reasoning": "General greeting"}
- "I’m getting an error installing requirements.txt" → {"needs_devrel": true, "priority": "high", "reasoning": "Direct technical setup issue"}
- "Who wants coffee?" → {"needs_devrel": false, "priority": "low", "reasoning": "Not development related"}
"""