Response to OMB Procurement of Artificial Intelligence RFI

April 29, 2024
View source
Other contributors:

The Machine Intelligence Research Institute (MIRI) has submitted a response to the Office of Management and Budget's Request for Information on Procurement of Artificial Intelligence. We focus on the potential risks associated with advanced AI systems, particularly those capable of causing societal-scale damage. We urge the OMB to be forward-looking in developing requirements and regulations that will adequately address challenges posed by future AI systems.

We consider AI systems capable of automating science and R&D, AI systems control of critical infrastructure, and AI systems, which if stolen by a malicious actor, could be repurposed to create chemical, biological, radiological, nuclear (CBRN) threats.Our submission highlights that the capabilities of large AI models are often unpredictable, both before and after training. We stress the need for robust risk assessment and management processes as part of the procurement process. However, we caution that there are currently no reliable methods for establishing whether an AI system is dangerous.

Our response outlines several recommendations, including:

  • Thorough evaluation of AI systems for potential unexpected harm.
  • Adequate cybersecurity measures to prevent theft by malicious actors.
  • Independent risk assessments by both vendors and agencies.
  • Public sharing of impact assessments when safe to do so.
  • Extensive safeguards during real-world testing of AI systems.
  • Ongoing monitoring and regular risk evaluations.
  • Adequate human oversight and accountability in AI decision-making.

Footnotes