Recursive self-improvement
noun
An AI system improving the process that builds the next, more capable version of itself, including choosing data, running training and checking the result.
Also known as RSI, self-improving AI
Recursive self-improvement is the possibility that an AI system could help build a more capable version of itself, then use that increased capability to improve the next one.
In the version discussed in the book, the system would choose data, run training and check the result without a human executing each step. The recursion is the loop: the improved tool becomes part of the machinery that produces a still better tool.
This is not the same as an ordinary software update, and it is not proof that such a loop will arrive on schedule. It is a precise version of the safety question: what happens when making the next machine is no longer work only humans can do? The honest answer in the chapter is that nobody knows.