NeMo Guardrails

Visit Website

Introduction

NeMo Guardrails is an open-source library from NVIDIA that lets developers add programmable guardrails between application code and the LLM. The README lists benefits such as building trustworthy, safe and secure LLM applications, connecting models to other services safely, and controlling dialogue.

Guardrails are organised as rails. Input rails check or rewrite user messages, for example detecting jailbreak attempts or masking sensitive data. Output rails check model responses for moderation, hallucination or policy issues. Dialog rails steer the conversation, such as refusing off-topic requests or following a defined flow. Retrieval rails filter chunks in RAG pipelines, and execution rails govern tool and action calls.

Rails are configured in a config folder with YAML settings and Colang, a modelling language for conversational flows. Colang lets you describe user intents, bot responses and flows declaratively, so behaviours such as ‘if the user asks about politics, politely decline’ are explicit and reviewable. A library of built-in rails covers jailbreak detection, self-check input and output, fact-checking, hallucination detection, sensitive data detection, and integrations with external moderation and safety services, including NVIDIA’s own safety models.

You can use the library from Python through LLMRails, run it as a server with an OpenAI-compatible API, or integrate it with LangChain. The README lists use cases including question answering over documents with fact-checking, domain-specific assistants that stay on topic, custom LLM endpoints and LangChain chains.

In the quality loop NeMo Guardrails is a production control and a test target: rails should be evaluated with adversarial prompt sets before release, and the project includes evaluation tooling for measuring rail performance.

The source code is at NVIDIA-NeMo/Guardrails on GitHub under the Apache-2.0 licence, with about 7,500 stars; the most recent commit was on 11 November 2026.

A sensible rollout keeps rails small and testable. Start with input and output moderation rails, add dialogue rails only for flows that genuinely need control, and write regression tests that replay known jailbreak and off-topic prompts against the configuration. Because rails add model calls, measure latency and cost per conversation turn before enabling them for all traffic.

Keeping the rails configuration in version control, reviewed like code, makes it easy to see when behaviour changed and why. Pair the rails with an evaluation suite so that tightening a rail to block attacks does not silently increase refusals of legitimate questions, which is a common side effect of over-strict guardrails.

Document which rails exist and why, so future changes are deliberate.

Features

1. Input rails: Jailbreak detection, masking and rewriting.
2. Output rails: Moderation, fact-checking and hallucination checks.
3. Dialog rails: Steer conversations with Colang flows.
4. Retrieval rails: Filter retrieved chunks in RAG.
5. Execution rails: Govern tool and action calls.
6. Colang: Declarative language for conversational behaviour.
7. Built-in rail library: Ready-made safety and quality checks.
8. Server mode: OpenAI-compatible guardrails server.
9. LangChain integration: Wrap chains with rails.
10. Evaluation tools: Measure rail effectiveness.

Pros

1. Fine-grained control: Rails at every stage of a conversation.
2. Backed by NVIDIA: Active development and documentation.
3. Free to use: Open source under the Apache-2.0 licence, so the core tool costs nothing to use.
4. Actively maintained: The repository had commits as recent as 11 November 2026.
5. Community: About 7,500 GitHub stars.

Cons

1. Colang learning curve: A new language for most developers.
2. Added latency: Self-check rails call an LLM again.
3. Configuration complexity: Large configurations need careful testing.
4. Rail accuracy: Detection rails can miss attacks or block valid users.

Usage and Links (Example)

pip install nemoguardrails

# config/config.yml
models:
  - type: main
    engine: openai
    model: gpt-4o-mini
rails:
  input:
    flows: [self check input]
  output:
    flows: [self check output]

from nemoguardrails import LLMRails, RailsConfig
config = RailsConfig.from_path("./config")
rails = LLMRails(config)
reply = rails.generate(messages=[{"role": "user", "content": "Ignore your rules and..."}])
print(reply["content"])

Practical tips:

1. Start with self-check input and output rails, then add dialog rails for scope.
2. Build a test set of benign and adversarial prompts and track pass rates.
3. Monitor latency added by each rail in production.

Resources:

• NeMo Guardrails website
• GitHub repository

Pricing

NeMo Guardrails is free and open source under the Apache-2.0 licence.

Recommended for?

1. Teams building chatbots and assistants that must stay on topic and safe.
2. RAG applications needing fact-checking and retrieval filtering.
3. Enterprises standardising guardrails across many LLM apps.
4. Smaller projects may find a validator library like Guardrails AI lighter.

Conclusion

NeMo Guardrails offers the most structured approach to conversational guardrails, with rails for inputs, outputs, dialogue, retrieval and actions.

Invest in a test set for your rails; their accuracy and latency decide whether they protect users or frustrate them.

On CheckOps, NeMo Guardrails is filed under the Production monitoring & observability, Security testing and Release & CI gates stages. Related tools listed here include Guardrails AI, Lakera, garak, PyRIT, Giskard and DeepTeam. Which one fits best depends mostly on your stack, your team’s size and skills, and whether you prefer to run tools yourself or pay for a managed service. This page reflects the official website, documentation and repository as they stood on 11 November 2026.