Adversarial machine learning is a technique aimed at deceiving the ML model by providing specially crafted input to fool the AV into classifying the malicious input as a benign file and evade detection. Like software, machine learning (including deep learning models) are susceptible to exploits as hackers seek to achieve their malicious objectives, like stealing data from users. 25 organizations surveyed did not know how to secure their machine learning-based systems, according to Deep Instinct. The company identified droppers used in a highly widespread Emotet attack that was able to routinely avoid detection by machine learning models.
Source: https://www.helpnetsecurity.com/2021/01/05/adversarial-machine-learning/