Managing Generative AI Risk and Meeting M-24-10 Mandates on Monitoring & Evaluation
OMB’s memo M-24-10 (5c. Minimum Practices for Safety-Impacting and Rights-Impacting Artificial Intelligence) is prescriptive (and timebound): No later than December 1, 2024 and on an ongoing basis while using new or existing covered safety-impacting or rights-impacting AI, agencies must ensure these practices are followed for the AI: D. Conduct ongoing monitoring. In addition to pre-deployment […]
Test & Evaluation Techniques for Meeting M-24-10 Mandates to Manage Generative AI Risk
Overview The release of the National Institute of Standards and Technology (NIST)’s AI Risk Management Framework (AI RMF) helped put a framework around how testing would enable organizations to manage and mitigate AI risks. While testing is predominantly considered a core part of model development, the NIST AI RMF emphasizes the importance of continuous testing […]