Integral Ad Science Scales Transparent and Compliant AI Products with AI Observability
Integral Ad Science (IAS) ensures superior model performance, compliance with industry standards, and better communication across stakeholders by monitoring their machine learning (ML) models using Fiddler AI Observability.
Results
With Fiddler, IAS has experienced significant benefits:
- Reduction in monitoring operation costs due to increased efficiency.
- Proactive regulatory compliance and easier demonstration of responsible AI practices to stakeholders.
- A unified, standardized view of rich model insights and metrics for improved team collaboration, and audits.
Committed to Responsible AI Governance
IAS, a leading global media measurement and optimization platform, delivers the industry’s most actionable data to drive superior results for the world’s largest advertisers, publishers, and media platforms.
Over time, IAS has increased its investment in AI in order to enhance the scope and speed of its digital ad measurement products and services. A key part of executing this strategy is establishing rigorous oversight and monitoring of the ML models powering their products in order to ensure quality and to build and maintain trust with their customers. Additionally, maintaining compliance with emerging AI regulatory requirements and industry standards is crucial for establishing and upholding their reputation as a responsible AI company.
To support their AI system monitoring and compliance goals, IAS chose Fiddler AI as one of their AI observability partners.
“One of the things that was appealing to IAS about Fiddler was its ability to customize the monitoring to specific model type, data volume and desired insights. Additionally, the dashboard views, automated alerting and ability to generate audit evidence also factored into the decision to work with Fiddler.”Kevin Alvero, Chief Compliance Officer, IAS
Sophisticated Model Monitoring and Dashboards for Transparency and Compliance
The IAS team has streamlined their MLOps practices as they look to continually ensure that their ML models are explainable, transparent, and compliant, reinforcing their commitment to responsible AI. Fiddler enables the IAS team to strengthen their AI governance and adhere to responsible AI practices by providing a unified view of their models and comprehensive audit trails for compliance and auditing purposes.
Not only can they monitor ML models and track metrics for audit and compliance, but they can also enhance communication and collaboration across the organization. They appreciate the ease of customizing dashboards and reports for each model, visualizing desired model insights tailored to stakeholder needs. When specified ML metrics fall outside acceptable thresholds, automated alerts are sent to the team, enabling them to do further analysis using Fiddler’s root cause analysis capability.
“At IAS, our entire product suite is powered through AI, machine learning and data driven decision making. The level of observability that the Fiddler AI Observability platform brings to our modeling stack has been extremely helpful in proactively monitoring and actioning on any gaps in the data coverage or any drifts in the input data, concept, or prediction that could impact the quality of our client facing solutions.”Kumaresh Singh, SVP Data Science, IAS
Using Fiddler’s customizable dashboards and reports, the IAS team provided rich insights on ML models and AI data tailored for various personas, including clients, auditors, and industry standards-setting bodies. Additionally, the dashboards provide a unified view of ML metrics that help facilitate cross-team communication within the organization to quickly resolve model issues and meet business KPIs, such as improving mean-time-to-identify (MTTI) and mean-time-to-respond (MTTR) rates.
Quick and Seamless Fiddler Implementation for Scalable ML Deployments
IAS found the implementation of the Fiddler AI Observability platform to be remarkably smooth and efficient. From the onset, the Fiddler team proactively guided IAS every step of the process and gave their expert advice on handling data volume, defining key metrics that show model performance, and setting up custom dashboards unique for the use case.
IAS was particularly impressed by the ease of integration, which stood out as one of the most seamless experiences in their extensive history of software implementations. The continuous support from Fiddler has not only guaranteed a successful rollout but also laid a solid foundation for onboarding additional models to scale their AI projects in the future.
“As we move forward, IAS is looking to continue to implement additional models onto the Fiddler platform as this will enable us to achieve consistency across format and process for monitoring. Additionally, this will provide management with a single location to access insights related to model performance and for the auditors to export evidence to support all audit efforts around our models running in production.”Kevin Alvero, Chief Compliance Officer, IAS
As an AI observability partner, Fiddler supports IAS to:
- Rapidly launch and scale AI: Establish a standardized platform for all stakeholders to gain oversight and continuously monitor ML models, enabling them to rapidly launch more AI-powered products at scale.
- Align with the highest standards of quality and transparency: Audit evidence of active and historical model monitoring to ensure compliance with internal policies and external standards, while providing transparency on how ML models drive customer metrics.
- Adhere to AI regulations: Demonstrate responsible AI by exercising model monitoring, explainability, and governance to assure ML models and applications are compliant with evolving AI regulations.