AI teams are often optimistic about the impact of their work, with the potential to transform industries from education to healthcare to transportation. But these teams must also consider possible negative consequences and implement systems to prevent harm. Parul Pandey, Principal Data Scientist at H2O.ai, joined us on AI Explained to offer expert tips and best practices for ML for high risk applications. Watch the webinar on-demand now and check out some of the key takeaways below.
Use the NIST AI Risk Management Framework as a Guide
AI regulations are in their early stages, with the NIST AI Risk Management Framework (RMF) offering a broad set of guidelines to enhance the trustworthiness of AI products. The NIST AI RMF broadly classifies risk into four categories - Govern, Map, Measure, and Mitigate - providing a comprehensive approach to cultivating a risk management culture, identifying and assessing risks, and mitigating their potential impact.
Implement Responsible AI Practices from Day One
In order to create trustworthy AI products, ML teams must adopt a holistic approach that integrates responsible AI practices throughout the entire MLOps lifecycle. Lessons can be learned from highly regulated industries such as banking, where strict fairness, model governance and model risk management practices have been around for quite some time. Model predictions, like loan or credit card approvals, need to be fair, unbiased, and transparent so that users receive equitable outcomes. Predicting possible failures in advance, through tools like an AI incident database, can help teams conserve resources while mitigating risks. Additionally, risk tiering allows for optimal workforce allocation, while robust documentation and clear usage guidelines enhance transparency. Regular checks for model drift and decay using a model monitoring platform helps ensure models work as intended after being deployed to production. Importantly, the implementation of independent testing teams can reveal potential issues overlooked by developers, improving model robustness and ensuring comprehensive evaluation.
Team Incentives are Critical for High-quality AI
Creating trustworthy AI models necessitates incentives that reward not only leadership but also teams such as data scientists for identifying potential issues. Dedicated teams for risk assessment and auditing can significantly contribute to model reliability. This concept is exemplified by Twitter's incident involving their biased image cropping algorithm, which led to the introduction of a 'bug bounty' to uncover additional issues. 'Bug bounties' were also implemented by organizations like OpenAI to help improve their model performance. Ultimately, the development of robust and trustworthy AI systems can lead to long-term returns, safeguarding against potential legal or reputational consequences. Therefore, the focus should extend beyond immediate business gains to include overall system integrity.
Model Explainability is Essential
Individuals interacting with AI models, particularly in high-stakes situations, want to comprehend the reasoning behind the AI's decisions. However, the opaque nature of some predictive models makes it difficult to understand their decision-making process. Despite the common belief in a trade-off between model accuracy and explainability, MLOps teams can use explainable AI to provide both to ensure transparency and regulatory compliance.
