The Machine Intelligence Research Institute (MIRI) has submitted a response to the Office of Management and Budget's Request for Information on Procurement of Artificial Intelligence. We focus on the potential risks associated with advanced AI systems, particularly those capable of causing societal-scale damage. We urge the OMB to be forward-looking in developing requirements and regulations that will adequately address challenges posed by future AI systems.
We consider AI systems capable of automating science and R&D, AI systems control of critical infrastructure, and AI systems, which if stolen by a malicious actor, could be repurposed to create chemical, biological, radiological, nuclear (CBRN) threats.Our submission highlights that the capabilities of large AI models are often unpredictable, both before and after training. We stress the need for robust risk assessment and management processes as part of the procurement process. However, we caution that there are currently no reliable methods for establishing whether an AI system is dangerous.
Our response outlines several recommendations, including: