NVIDIA and Fiddler: Accurate and Safe LLM Applications
Secure LLM Applications with NVIDIA and Fiddler
NVIDIA NeMoTM Guardrails is a scalable LLM guardrail orchestration platform that enables organizations to implement and orchestrate multiple AI guardrails, ensuring the safety, security, accuracy, and topical relevance of their LLM interactions across popular generative AI frameworks.
Fiddler offers an out-of-the-box integration with NeMo Guardrails, enabling enterprises to leverage the industry’s fastest guardrails to safeguard LLM applications against hallucinations, toxicity, and jailbreak attempts — all within their secure NeMo Guardrails environment.
Fiddler Guardrails is powered by the Fiddler Trust Service, which is comprised of purpose-built, fine-tuned Fiddler Trust Models designed for high accuracy scoring of LLM prompts and responses with low latency.
Analyze Data from NeMo Guardrails in Fiddler
Fiddler enables enterprises to bring all prompts, responses, metadata, and executed rails from NeMo Guardrails into the Fiddler AI Observability and Security platform — regardless of the guardrail type or provider. Data is then visualized in the Fiddler platform using customized dashboards and reports, providing AI teams with a unified view of their LLM applications.
- Gain contextual insights into rails execution
- Define rich alerts for rail activities
- Detect hallucination, safety violations, prompt injection attacks, and operational issues
- Perform root cause analysis on LLM performance issues for quick resolution
Monitor NVIDIA NIM Inference-Level Logs with Fiddler
NVIDIA NIM accelerates the deployment of AI foundation models by offering portable, quick inference capabilities with minimal setup. By packaging AI models into containers, the NIM microservice simplifies integration into existing Generative AI infrastructure with minimal code changes.
When integrated with the Fiddler AI Observability and Security Platform, NVIDIA NIM ensures that LLM deployments scale efficiently while remaining secure. Prompts processed by NIM are logged and automatically routed to Fiddler’s platform, enabling real-time guardrailing and monitoring, actionable insights, and enhanced operational oversight.
Featured Resources
What is Fiddler Auditor?
Fiddler Auditor is the open-source robustness library for red-teaming of LLMs. Evaluate the robustness, correctness, safety, and privacy of LLMs and prompts in pre-production.
