Response to NTIA AI Open Model Weights RFC

March 27, 2024
View source
Other contributors:

The Machine Intelligence Research Institute (MIRI) has submitted a response to the NTIA Request for Comment on AI Open Model Weights. In this response, we emphasize the potential catastrophic risks associated with advanced AI systems, particularly those that could surpass human capabilities.The submission highlights that while current AI systems likely don't pose catastrophic risks due to limited capabilities, future systems may become significantly more dangerous. We point out that there's no reliable method to determine AI capabilities before training or to rule out unexpected capabilities after training.

We further stress that releasing model weights is irreversible and could enable malicious actors to cause harm.We make recommendations around understanding and mitigating risks before AI systems are deployed or released. There is a trend of model weights becoming widely available, either through intentional releases or leaks. Monitoring AI usage can reduce risks, but this becomes impossible if weights are widely available. There is a need for robust evaluations to determine the safety of releasing model weights. Current metrics for regulating AI development, such as computational resources used, may become inadequate and need frequent adjustment.

Footnotes