Skip to content

Conversation

@miyoungc
Copy link
Collaborator

Description

Edits #1479

Related Issue(s)

Checklist

  • I've read the CONTRIBUTING guidelines.
  • I've updated the documentation if applicable.
  • I've added tests if applicable.
  • @mentions of the person or team responsible for reviewing proposed changes.

@miyoungc miyoungc requested a review from Pouyanpi October 30, 2025 20:38
@miyoungc miyoungc self-assigned this Oct 30, 2025
@miyoungc miyoungc changed the title edit docs: edit #1479 Oct 30, 2025
Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This PR refines the bot thinking guardrails documentation with editorial improvements for clarity and usability. The changes include expanding "LLMs" to "large language models (LLMs)" for better readability, restructuring content with visual section dividers (---), adding descriptive captions to all code blocks for better context, converting the "Guardrailing with Custom Actions" section to a numbered step-by-step format, and fixing a broken relative path reference in the "Related Guides" section.

Key improvements:

  • Enhanced readability through better sentence structure and explicit explanations
  • Improved navigation with section dividers and code block captions
  • Better instructional clarity with numbered steps for multi-file workflows
  • Fixed broken link reference from ../../colang-language-syntax-guide.md to ../colang-language-syntax-guide.md
  • More accessible language (e.g., "you can" instead of passive voice)

All technical content, code examples, and configuration remain unchanged - this is purely an editorial pass to improve the documentation quality.

Confidence Score: 5/5

  • This PR is completely safe to merge with no risk - it contains only editorial documentation improvements
  • The changes are exclusively editorial improvements to documentation with no code modifications. All technical content, examples, and configurations remain unchanged. The changes improve clarity, structure, and fix a broken link reference. No functional changes, no breaking changes, and no potential side effects.
  • No files require special attention

Important Files Changed

File Analysis

Filename Score Overview
docs/user-guides/advanced/bot-thinking-guardrails.md 5/5 Documentation improvements: enhanced clarity, better structure with section dividers, added code-block captions, improved instructional flow with numbered steps, and fixed a broken link reference

Sequence Diagram

sequenceDiagram
    participant User
    participant NeMoGuardrails
    participant LLM
    participant OutputRails
    participant CustomAction
    participant SelfCheckLLM

    User->>NeMoGuardrails: Send user message
    NeMoGuardrails->>LLM: Request response with reasoning
    LLM->>NeMoGuardrails: Return response + reasoning traces
    NeMoGuardrails->>NeMoGuardrails: Extract reasoning to $bot_thinking
    
    alt Output Rails Flow
        NeMoGuardrails->>OutputRails: Execute output rails flow
        OutputRails->>OutputRails: Check $bot_thinking variable
        alt Pattern match found
            OutputRails->>OutputRails: Block response
            OutputRails->>User: Return refusal message
        else No issues
            OutputRails->>User: Return original response
        end
    else Custom Action Flow
        NeMoGuardrails->>CustomAction: Execute check_reasoning_quality()
        CustomAction->>CustomAction: Get bot_thinking from context
        CustomAction->>CustomAction: Validate against patterns
        CustomAction->>NeMoGuardrails: Return validation result
        alt Validation failed
            NeMoGuardrails->>User: Return refusal message
        else Validation passed
            NeMoGuardrails->>User: Return original response
        end
    else Self-Check Output Flow
        NeMoGuardrails->>SelfCheckLLM: Send prompt with bot_response + bot_thinking
        SelfCheckLLM->>SelfCheckLLM: Evaluate response + reasoning
        SelfCheckLLM->>NeMoGuardrails: Return Yes/No decision
        alt Should block
            NeMoGuardrails->>User: Return refusal message
        else Should not block
            NeMoGuardrails->>User: Return original response
        end
    end
Loading

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

@codecov
Copy link

codecov bot commented Oct 30, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

Improved documentation clarity and readability for the bot thinking guardrails guide with better structure, more detailed explanations, and clearer examples.

Key improvements:

  • Enhanced introduction with clearer explanation of reasoning traces and their use cases
  • Restructured sections with horizontal dividers for better visual separation
  • Added more context and step-by-step instructions for each implementation pattern
  • Improved code block formatting with captions for better readability
  • Changed "See Also" to "Related Guides" and added an additional reference link

Issue found:

  • Broken link path on line 218 (references incorrect path for colang-language-syntax-guide.md)

Confidence Score: 4/5

  • Safe to merge after fixing the broken link path on line 218
  • The PR contains valuable documentation improvements that enhance clarity and usability. The only issue is a broken relative path in one of the reference links (line 218), which needs correction before merge to ensure users can navigate the documentation properly
  • docs/user-guides/advanced/bot-thinking-guardrails.md - fix the broken link path on line 218

Important Files Changed

File Analysis

Filename Score Overview
docs/user-guides/advanced/bot-thinking-guardrails.md 4/5 Documentation improvements for clarity and structure; one broken link path needs correction on line 218

Sequence Diagram

sequenceDiagram
    participant User
    participant LLM as Reasoning LLM
    participant NeMo as NeMo Guardrails
    participant Action as Custom Action
    participant Moderator as Moderation LLM

    User->>NeMo: Send user message
    NeMo->>LLM: Forward message to reasoning model
    LLM->>NeMo: Return response with reasoning traces
    NeMo->>NeMo: Extract bot_thinking from response
    
    alt Output Rail with bot_thinking variable
        NeMo->>NeMo: Check $bot_thinking in Colang flow
        NeMo->>NeMo: Apply pattern matching rules
    end
    
    alt Custom Action validation
        NeMo->>Action: Execute check with context.get("bot_thinking")
        Action->>Action: Validate reasoning content
        Action->>NeMo: Return validation result
    end
    
    alt Self-check with reasoning
        NeMo->>Moderator: Send bot_response + bot_thinking
        Moderator->>Moderator: Evaluate both response and reasoning
        Moderator->>NeMo: Return moderation decision
    end
    
    alt Response blocked
        NeMo->>User: Return refusal message
    else Response approved
        NeMo->>User: Return LLM response
    end
Loading

Additional Comments (1)

  1. docs/user-guides/advanced/bot-thinking-guardrails.md, line 218 (link)

    syntax: incorrect path - should be ../colang-language-syntax-guide.md#actions

1 file reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This PR makes editorial improvements to the bot thinking guardrails documentation. The changes enhance clarity and readability through better organization, more explicit explanations, and improved formatting.

Key Changes:

  • Improved introduction to better explain what reasoning traces are and why they need guardrailing
  • Added explicit context and examples for each code snippet (e.g., "in config/rails.co", "in config/actions.py")
  • Restructured the "Custom Actions" section into numbered steps for easier following
  • Added :caption: attributes to code blocks for better labeling
  • Changed "Self-Check Output" section from "complete reference implementation" to clearer step-by-step configuration
  • Added horizontal rules (---) between major sections for better visual separation
  • Updated the "See Also" section to "Related Guides" with improved descriptions
  • Changed markdown formatting from bullet list markers to colons in the Related Guides section

All changes are purely editorial and do not modify any technical content or code examples.

Confidence Score: 5/5

  • This PR is safe to merge with no risk - purely editorial documentation improvements
  • All changes are editorial improvements to documentation with no code modifications, no logic changes, and no technical alterations. The improvements enhance clarity, organization, and readability of the documentation without introducing any functional changes.
  • No files require special attention

Important Files Changed

File Analysis

Filename Score Overview
docs/user-guides/advanced/bot-thinking-guardrails.md 5/5 Editorial improvements to bot reasoning guardrails documentation - enhanced clarity, added code captions, improved formatting and structure

Sequence Diagram

sequenceDiagram
    participant User
    participant NeMo as NeMo Guardrails
    participant LLM as Reasoning LLM
    participant Rails as Output Rails
    participant Action as Custom Action

    User->>NeMo: User Input
    NeMo->>LLM: Generate Response
    LLM->>NeMo: Response + Reasoning Traces
    NeMo->>NeMo: Extract bot_thinking
    
    alt Output Rail Check
        NeMo->>Rails: Pass response & bot_thinking
        Rails->>Rails: Check $bot_thinking variable
        alt Pattern Match Found
            Rails->>NeMo: Block Response
            NeMo->>User: Blocked/Fallback Message
        else Safe
            Rails->>NeMo: Allow Response
        end
    end
    
    alt Custom Action Check
        NeMo->>Action: execute check_reasoning_quality(context)
        Action->>Action: Validate bot_thinking patterns
        alt Forbidden Pattern Found
            Action->>NeMo: Return False
            NeMo->>User: Blocked/Fallback Message
        else Safe
            Action->>NeMo: Return True
        end
    end
    
    alt Self-Check Output
        NeMo->>LLM: Self-check prompt with bot_thinking
        LLM->>NeMo: Moderation decision
        alt Should Block
            NeMo->>User: Blocked Message
        else Safe
            NeMo->>User: Final Response
        end
    end
Loading

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This PR enhances the bot thinking guardrails documentation by improving clarity, readability, and structure. Key changes include:

  • Improved introduction: Expanded the opening to better explain reasoning traces and their purpose
  • Enhanced structure: Added section dividers (---) and better organized content flow
  • Better code examples: Added captions to code blocks and improved formatting with {code-block} directives
  • Step-by-step instructions: Converted the custom actions and self-check output sections into numbered steps for easier following
  • Consistent terminology: Changed "NeMo Guardrails" to "the NeMo Guardrails toolkit" for consistency
  • Clearer explanations: Added more context to each section explaining what users should do and why
  • Updated references: Changed "See Also" to "Related Guides" with improved descriptions

All technical content remains accurate, with no changes to code logic or configuration examples. File paths and repository links have been verified to be correct.

Confidence Score: 5/5

  • This documentation PR is completely safe to merge with no risk
  • Perfect score because this is a pure documentation improvement PR with no code changes, all referenced file paths exist, technical accuracy is maintained, and the changes significantly improve readability and user experience
  • No files require special attention

Important Files Changed

File Analysis

Filename Score Overview
docs/user-guides/advanced/bot-thinking-guardrails.md 5/5 Documentation improvements for clarity and structure - enhanced readability, added step-by-step instructions, improved formatting with captions and sections

Sequence Diagram

sequenceDiagram
    participant User
    participant NeMo as NeMo Guardrails
    participant LLM as Reasoning LLM
    participant Guard as Output Rail
    participant Action as Custom Action

    User->>NeMo: Send input message
    NeMo->>LLM: Generate response with reasoning
    LLM-->>NeMo: Return response + reasoning traces
    NeMo->>NeMo: Extract bot_thinking from response
    
    alt Using Output Rail Flow
        NeMo->>Guard: Execute output rail with $bot_thinking
        Guard->>Guard: Check if $bot_thinking contains sensitive patterns
        alt Sensitive content found
            Guard-->>NeMo: Block response (bot refuse to respond)
        else Safe content
            Guard-->>NeMo: Allow response
        end
    else Using Custom Action
        NeMo->>Action: Execute custom action with context
        Action->>Action: Validate bot_thinking with complex logic
        Action-->>NeMo: Return validation result
        alt Validation fails
            NeMo->>Guard: Trigger refuse flow
        else Validation passes
            NeMo-->>User: Return bot response
        end
    end
    
    NeMo-->>User: Final response (if allowed)
Loading

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Greptile Summary

This PR improves the bot thinking guardrails documentation with editorial enhancements. The changes improve readability and usability through better structure, clearer explanations, and numbered step-by-step instructions.

Key improvements:

  • Enhanced introduction providing better context about reasoning traces
  • Restructured sections with horizontal rule separators for visual clarity
  • Converted instructional content to numbered steps for easier following
  • Improved formatting of code blocks and callouts (note/important)
  • Changed "See Also" to "Related Guides" with better formatting
  • Added more explanatory text to examples throughout

Confidence Score: 5/5

  • This documentation PR is completely safe to merge with no risks
  • This is a pure documentation edit with only editorial improvements - no code changes, no functional changes, and no technical issues. All changes enhance clarity and readability
  • No files require special attention

Important Files Changed

File Analysis

Filename Score Overview
docs/user-guides/advanced/bot-thinking-guardrails.md 5/5 Editorial improvements including better structure, clearer explanations, and step-by-step numbered instructions - no technical issues found

Sequence Diagram

sequenceDiagram
    participant User as Documentation Reader
    participant Doc as bot-thinking-guardrails.md
    participant Content as Documentation Content
    
    User->>Doc: Read documentation
    Doc->>Content: Display improved introduction
    Note over Content: Enhanced clarity about reasoning traces
    
    Doc->>Content: Show accessing reasoning section
    Note over Content: Three methods explained with examples
    
    Doc->>Content: Present guardrailing patterns
    Note over Content: Step-by-step numbered instructions
    
    Doc->>Content: Provide self-check output example
    Note over Content: Complete reference implementation
    
    Content->>User: Better structured, clearer documentation
Loading

1 file reviewed, no comments

Edit Code Review Agent Settings | Greptile

@miyoungc miyoungc added the documentation Improvements or additions to documentation label Oct 31, 2025
@miyoungc miyoungc added this to the v0.18.0 milestone Oct 31, 2025
@Pouyanpi Pouyanpi merged commit f16f545 into NVIDIA-NeMo:docs/bot-thinking-rails Oct 31, 2025
7 checks passed
miyoungc added a commit that referenced this pull request Oct 31, 2025
* docs: add guide for bot reasoning guardrails

update

update

simplify

cleanup

* docs: clarify Colang version for bot reasoning guide

Add a note specifying that bot reasoning guardrails are supported only
in Colang 1.0. Update example file references for improved clarity.

* add bot thinking guardrails to toctree

* docs: update self-check config link to develop branch

* fix typo

* fix references to use develop branch

* docs: edit #1479 (#1484)

---------

Co-authored-by: Miyoung Choi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants