Verification and Validation for AI Systems

Overview

Artificial intelligence has successfully been used to solve complex problems where traditional methods have failed. However, there is a trade-off between predictive power and explainability of the methods used. While deep neural networks (DNNs) models usually cannot be explained or interpreted by their users, they can provide better results than classical machine learning and traditional methods for many tasks, which is leading to a desire to put such complex models in production in safety critical situations through the application of Validation and Verification (V&V) methods.

By integrating the AI components into models, you can perform system-level simulations to ensure requirements are satisfied, and therefore deployable to the desired target platform(s).

The presentation will also outline how Veoneer (Autoliv) successfully labeled Lidar data for verification of a Radar-based automated driving system. The output of the labeling process was used to train deep neural networks to provide a fully automated way to produce vehicle objects of interest which can be used to find false-negative events. This provided substantial benefit to their validation process to verify their Radar sensors. 

Highlights

A list of issues related to V&V for AI discussed include:

• Explainability: Can you explain the working of the AI system in human-understandable terms? 
• Interpretability:  Can you observe and trace cause and effect in an AI system and explain the rationale of the decision making? 
• Robustness: Is the AI system immune from spoofing and other common attacks to process reliable inputs? Adversarial inputs can help determine how robust an AI algorithm is.  
• Safety Certification: Has the AI system been developed with safety lifecycle as key component? 

About the Presenter

Emmanuel is an application engineer at MathWorks who first joined the company as a training engineer. He taught several MATLAB, Simulink and Simscape courses as well as specialized topics such as machine learning, statistics, optimization, image processing and parallel computing. Prior to joining MathWorks, he was a Lecturer in Mechatronic Engineering at the University of Wollongong. He holds a PhD in Mechanical Engineering from Virginia Tech. He also worked as a Systems / Controls Engineer at Cummins Engine Company and as a research assistant in several research institutions in California and Virginia