Robotics has produced impressive results through specialized controllers, imitation learning, reinforcement learning, and increasingly, vision-language-action models. Yet compared with the breadth of capability available through LLMs, useful general-purpose robotics still feels frustratingly out of reach. For us, that gap raises a question: how much more of the LLM approach can robotics inherit?
One response is that these are fundamentally different problems. Language has discrete tokens; physical action is continuous. Text comes with an enormous pretraining corpus; robotic experience does not arrive in the same convenient form. Robots must act on the world, not merely predict a continuation.
These are reasonable observations. The mistake would be to move directly from that list to the conclusion that robotics requires an entirely different learning architecture.
Behind LLMs sits an enormous accumulation of brainhours: experiments in optimization, scaling, pretraining, post-training, distillation, and inference. We want to reuse that work, including the lessons from experiments that failed. A familiar learning formulation also gives us a basis for adapting future improvements. Abandoning it prematurely means giving up more than a model architecture.
Tacitus’s software thesis is therefore to transmute the robotics problem into an LLM problem. We want to examine the differences one by one and ask whether they really prevent us from using the same successful machinery.
Some may be learning problems mistaken for architectural barriers: capabilities that sufficiently expressive models can acquire through richer experience and scale.
Other differences are real, but changeable. Continuous motion does not mean the model must operate without discrete tokens. GPC demonstrates this by learning a discrete motion vocabulary for a physically simulated character. The underlying physics remains continuous; the representation presented to the generative model changes. That is the kind of transformation we mean.
Robotics already borrows from foundation models. Our bet concerns how far we can take that approach: preserving the core of discrete autoregressive next-token learning while making physical experience amenable to it.
Our first focus is dexterous manipulation: a hand that can acquire and reuse capabilities across tasks.
— Matthew