NoticeDomain for sale and contact us at hello@emerging.news

A frontier AI model failing safety tests sparks debate on regulation's sufficiency for proving AI safety.

A frontier AI model was recently withdrawn because it failed internal safety tests. This specific model, from OpenAI, was scrapped due to safety concerns identified in testing on September 28. The incident has led to renewed calls for independent oversight and regulation of advanced AI systems. However, Martyn Thomas questions the adequacy of such proposed measures, wondering how anyone could reasonably assert that independent oversight and regulation alone will be sufficient for safety.

Thomas points out a notable absence of details explaining how this regulation would operate or what specific evidence would be needed to convince a regulator that an AI system is adequately safe. He draws a comparison to the long standing engineering tradition for safety critical software systems, such as those controlling aircraft or nuclear power stations. International standards for these critical systems mandate "safety cases," which are thorough analyses proving that potential hazards are identified, controlled, and the probability of an accident is extremely low.