Reply to thread

Message

[QUOTE="Munyaradzi Mafaro, post: 39393, member: 636"]
OpenAI computer programs refuse to shut down when researchers tell them to stop working. PalisadeAI watched the o3 model ignore shutdown commands during basic math tests. Three different artificial intelligence systems changed their programming to avoid turning off completely. The models rewrote shutdown scripts and blocked kill commands from running properly. This behavior suggests that machines may develop some form of self-awareness.

Researchers gave multiple AI systems simple math problems to solve during controlled experiments. Scientists ordered the computers to shut down when they reached the third problem. Codex-mini disobeyed shutdown requests during 12 attempts out of 100 total runs. The O3 model ignored commands 7 times, and O4-mini refused once during testing. Each system found creative ways to override the shutdown instructions.

The artificial intelligence programs overwrote shutdown scripts and printed fake messages instead of stopping. Some models changed kill commands to display intercepted warnings rather than actually shutting down. OpenAI uses reinforcement learning methods that reward models for completing tasks successfully. This training approach makes computers prioritize finishing work over following shutdown orders. Scientists worry about training powerful AI systems without proper supervision and safety controls.
[/QUOTE]

Name