NVIDIA and Fiddler: Safe and Trustworthy LLM Applications
Scalable LLM Applications with NVIDIA and Fiddler
Productionize Enterprise LLM Applications with NVIDIA NIM and Fiddler
NVIDIA NIM accelerates the deployment of AI foundation models by offering portable, quick inference capabilities with minimal setup. Its optimized microservices enable enterprises to deploy powerful generative AI models at scale, delivering low-latency, high-throughput inferences through engines like TensorRT and TensorRT-LLM. By packaging AI models into containers, NIM simplifies integration into existing Generative AI infrastructure with minimal code changes.
When integrated with the Fiddler AI Observability Platform, NVIDIA NIM ensures that LLM deployments scale efficiently while remaining secure and trustworthy. Prompts processed by NIM are logged and automatically routed to Fiddler’s platform, enabling real-time guardrailing and monitoring, actionable insights, and enhanced operational oversight within a comprehensive Generative AI infrastructure.
Ensure LLM Safety with NVIDIA NeMo Guardrails and Fiddler
NeMo Guardrails provides developers with a rich set of rails to moderate and protect conversations. Developers can define the behavior of LLM applications on specific topics and steer LLMs along pre-defined conversational paths for trustworthy and secure dialog.
Once integrated with Fiddler, the prompts, responses, metadata, and rails executed in NeMo Guardrails are published to the Fiddler AI Observability platform. This enables AI and app engineers to:
- Gain insights into rails execution
- Define rich alerts for rail activities
- Detect hallucination, drift, safety, and operational issues
Fiddler’s custom dashboards and reports provide developers with deep insights into LLM and rail metrics, enabling root cause analysis and effective issue resolution.
