[draft]
Current LLMs are only getting bigger and more expensive.
We predict 2 aspects of human-level intelligence will appear in future models:
-
Constant-sized attention windows. LLMs become slower as they gather more context, and at a certain point they must pause to compact what they've seen. We predict an entirely new architecture that is capable of streaming inputs and outputs in real time.
-
Learning through exploration. Current LLMs are fully guided when they learn and cannot choose what data to explore themselves. This encourages them to get the correct answer without reusing abstractions the same way a human would, making them less general and giving them "spiky" abstractions. [For example, you can ask them to design an image, and they will critique it like it was written by someone else, as if none of the critiques even occurred to them until now.]
As models become "more general," one can expect modularity of parts of the brain because they are able to take advantage of patterns in a more efficient way.
We predict the next 10 years will bring a "brain in a box" that is able to locally control any robot and quickly modify its weights to become dexterous at any task, and that is cheaper than a coffee.
In such a world, we predict there will be a large application layer of affordable robots -- both humanoid and specific-task machines.