Counterfit is a command-line tool for testing the security of its own AI systems and assessing them for vulnerabilities. Microsoft has decided to open-source it to help organizations verify that the algorithms they use are ‘robust, reliable, and trustworthy’ Counterfit started as a collection of attack scripts written to target individual AI models, but Microsoft turned it into an automation tool to attack multiple AI systems at scale. The tool is environment agnostic: the assessed AI models can be hosted in a cloud environment, on-premises, or on the edge of the cloud.
Source: https://www.helpnetsecurity.com/2021/05/05/security-testing-ai-systems/