The National Institute of Standards and Technology (NIST) released a new open-source software tool for testing the resilience of machine learning (ML) models to various types of attacks.
The tool, known as Dioptra, was released Friday along with new AI guidance from NIST marking the 270th day since President Joe Biden’s Executive Order on the Safe, Secure and Trustworthy Development of AI was signed.
The Dioptra tool, which is available on GitHub, will fulfil the executive order’s requirement for NIST to assist with AI model testing and also supports the “measure” function of NIST’s AI Risk Management Framework.
“Open source development of Dioptra started in 2022, but it was in an alpha ‘pre-release’ state until last Friday, July 26,” a NIST spokesperson told SC Media. “Key features that are new from the alpha release include a new web-based front end, user authentication, and provenance tracking of all the elements of an experiment, which enables reproducibility and verification of results.”
Free Dioptra AI testing platform measures impact of 3 attack categories
Previous NIST research identified three main categories of attacks against machine learning algorithms: evasion, poisoning and oracle.
Evasion attacks aim to trigger an inaccurate model response by manipulating the data input (for example, by adding noise), poisoning attacks aim to impede the model’s accuracy by altering its training data, leading to incorrect associations, and oracle attacks aim to “reverse engineer” the model to gain information about its training dataset or parameters, according to NIST.
The Dioptra tool was originally built to measure attacks against image classification models but could also be adapted to test other ML applications such as speech recognition models.
The free platform enables users to determine to what degree attacks in the three categories mentioned will affect model performance and can also be used to gauge the use of various defenses such as data sanitization or more robust training methods.
The open-source testbed has a modular design to support experimentation with different combinations of factors such as different models, training datasets, attack tactics and defenses.
Interactive web interface caters to range of user skill levels
The newly released 1.0.0 version of Dioptra comes with a number of features to maximize its accessibility to first-party model developers, second-party model users or purchasers, third-party model testers or auditors, and researchers in the ML field alike.
Along with its modular architecture design and user-friendly web interface, Dioptra 1.0.0 is also extensible and interoperable with Python plugins that add functionality. Dioptra also comes with documentation and demos that can help users with little programming experience familiarize themselves with Dioptra experiments.
Dioptra tracks experiment histories, including inputs and resource snapshots that support traceable and reproducible testing, which can unveil insights that lead to more effective model development and defenses.
The tool can be deployed in a multi-tenant environment to facilitate sharing of resources and components between users, but is also amenable to being deployed on a single local machine.
Dioptra is most compatible with Unix-based operating systems, such as Linux or MacOS, and experiments typically require significant computational resources, with the Dioptra architecture having been officially tested on an NVIDIA DGX server with 4 graphics processing units (GPUs).
“User feedback has helped shape Dioptra and NIST plans to continue to collect feedback and improve the tool,” a NIST spokesperson told SC Media.
NIST advances AI safety objectives with new published guidance
Publication of the Dioptra software package was also accompanied Friday with the release of a new draft document from NIST’s AI Safety Institute, which is focused on risk management for “dual-use” foundation models that could be leveraged for both positive and harmful purposes.
NIST will be accepting public comments on this guidance document until Sept. 9.
Additionally, NIST has published three final guidance documents that were previously published as draft documents.
The first tackles 12 unique risks of generative AI along with more than 200 recommended actions to help manage these risks. The second outlines “Secure Software Development Practices for Generative AI and Dual-Use Foundation Models,” and the third provides a plan for global cooperation in the development of AI standards.
“For all its potentially transformational benefits, generative AI also brings risks that are significantly different from those we see with traditional software. These guidance documents and testing platform will inform software creators about these unique risks and help them develop ways to mitigate those risks while supporting innovation,” NIST Director Laurie E. Locascio said in a statement.
