The original note, as written at the time
Here’s my suggestion in the trajectory for AI. We need AI, especially at future general intelligence. I’m a sceptic about superintelligence. Help us achieve a higher mechanised state. The higher mechanised state will involve machines that would have otherwise not been able to create, but machines which are not reliant on the AI to operate.
Ongoing development
Here is my suggestion for the trajectory of AI, and I want to be clear that I am a sceptic about superintelligence — not about capability, but about the specific claim that a very large model become something categorically different.
What we actually need is a higher mechanised state. By that I mean machinery that can do what machinery could never previously do, and — this is the part that defines it — machinery that is not dependent on an AI to operate.
Think about the difference. A robot arm that requires a language model in a data centre to decide its next movement is not a machine. It is a terminal. Cut the network and it stops, and everything it has learned about how to behave in the world is somewhere else, owned by someone else, priced monthly.
A machine in a higher mechanised state carries its competence. It is competent in the way a good engine is competent, and when it is finished learning, the learning is in the object, and the object works in a village with no connection. It can be repaired by a person who understands mechanisms rather than a person who understands APIs.
That is the opposite direction from the one we are travelling. We are centralising capability into enormous models accessible only by connection, and calling the resulting dependency a service. Every advance that requires a subscription is not an advance in mechanisation. It is an advance in rent.
So the test I would apply to any AI development is this: does it produce a machine that could survive the disappearance of the model? If yes, the capability has been transferred. If no, we have only moved the intelligence somewhere less reachable and arranged to bill for access.
I would rather have fewer spectacular systems and a great deal more machinery that knows how to do one thing, on its own, anywhere, for twenty years.