TTT and Functional Plasticity

Recent research has made online weight updates increasingly practical. Linear fast-weight models such as DeltaNet and Falcon update associative memory through local prediction errors. TTT, Titans, and ATLAS extend online learning to nonlinear neural memory.

In my previous post on foundation model plasticity, I reviewed these approaches and their limitations.

The progress raises a more fundamental question: what should a model learn from its current experience to improve its ability to solve future tasks?

Functional Plasticity refers to the ability of a foundation model to acquire useful, transferable capabilities through experience-driven parameter updates, rather than merely fitting new data or storing new associations.

1. What Makes Experience Worth Learning?

Every learning algorithm needs a criterion for deciding which changes are useful. Supervised learning uses a prediction objective. Reinforcement learning typically uses expected returns.

For a general-purpose agent, future tasks may be unknown when learning takes place.

There is a fundamental logical limit: if future tasks and environments are completely arbitrary, no learning rule can guarantee that every update will be useful. With finite model capacity (e.g., the model size and the compute budget), deciding what to retain requires assumptions about future relevance.

However, uncertainty about future goals and uncertainty about the world are different problems.

The first case is relatively well understood. An agent can learn how its environment works before knowing what it will be asked to do.

The second case is much harder. Knowledge acquired in one world must help the agent learn another. This requires abstractions and learning mechanisms that remain useful across changing environments.

This gives us two possible foundations for functional plasticity:

• Transferable representations of environment dynamics capture structures that support different goals, including causal relationships, state abstractions, and temporal abstractions.

• Meta-skills are reusable procedures for exploration, hypothesis testing, and learning new tasks.

The central research question becomes: how can these forms of knowledge emerge from experience and become part of the model's persistent capabilities?

2. TTT’s Inner and Outer Loops

TTT treats the hidden state of a sequence model as a trainable model whose parameters are updated during inference.

• Its Inner Loop performs local self-supervised learning:

$$\theta_{t+1}=\theta_t-\eta\nabla_\theta\ell_\phi(\theta_t;x_t)$$

Here, $\theta_t$ denotes the memory parameters before the update on layer input $x_t$. The slow outer-loop parameters $\phi$ include the projections that define the local task, backbone parameters, and any learned initialization and update settings. These slow parameters are fixed during online inference.

TTT-Linear updates a linear mapping. TTT-MLP updates a two-layer neural network, allowing more expressive online adaptation.

• The Outer Loop learns the projections that define the local learning task, along with the other slow parameters, using the overall language modeling objective.

For sampled training token sequences $\mathcal S=(s_1,\ldots,s_{T+1})$, the outer objective can be written schematically as:

$$\phi^*=\arg\min_\phi\mathbb E_{\mathcal S} \left[\sum_{t=1}^{T}-\log p_\phi \left(s_{t+1}\mid s_{\leq t};\theta_{1:t+1}(\phi;\mathcal S)\right)\right].$$

Here, $s_t$ is a token, while $x_t$ is an input representation at a TTT layer. The trajectory $\theta_{1:t+1}(\phi;\mathcal S)$ starts from the learned initialization $\theta_1(\phi)$ and follows the inner updates; the prediction distribution also depends directly on $\phi$. For $t=1,\ldots,T$, the input $x_t$ corresponds to token $s_t$, and its inner update produces $\theta_{t+1}$. Outer-loop training differentiates through these updates, including their inner gradients.

• As we can see here, TTT uses the language-modeling objective to shape local learning.

The Inner Loop determines how parameters change from experience. The Outer Loop trains the conditions under which those changes improve language modeling.

This raises the next question: how should the Outer Loop determine which learning is useful?

This question is important since the outer loop optimizes next-token prediction on the training distribution. For a general-purpose agent, the relevant future objectives may be unknown during training.

3. Plasticity Capacity vs. Functional Plasticity

I think existing approaches should be evaluated along two dimensions.

• Plasticity Capacity: What can a foundation model change through online parameter updates?

• Functional Plasticity: What new, useful, and transferable capabilities do these changes produce?

In summary, Plasticity Capacity determines what a model can change. Functional Plasticity determines whether those changes make it a better learner.

4. What Should Be Learned for Future Unknown Tasks?

Consider a robot that discovers that pressing a button opens a door.

• It could learn the button's location, the action needed to press it, or the causal relationship between the button and the door.

• It could also acquire a general procedure: identify controllable factors, intervene, and examine whether the reachable space changes.

These forms of knowledge have different potential transfer scopes.

Representations of Environment Dynamics

Learning the structure of the world before knowing the goal is a promising starting point.

In Reward-Free Exploration for Reinforcement Learning, an agent explores an MDP before receiving a reward function. The collected information can support planning for different rewards in the same environment.

This work’s result illustrates an important principle: an agent can acquire useful knowledge before its future objective is specified.

For general agents, candidate abstractions include causal relationships, action-conditioned predictions, controllability, reachability, and temporal structure.

The challenge is to discover compact representations that preserve the consequences of interventions across different goals.

Meta-skills

When the environment itself changes, specific world knowledge may lose its usefulness.

The agent must discover new dynamics, test hypotheses, identify relevant variables, and construct new plans.

These learning procedures may be transferable even when the environments share few visual or semantic similarities.

I think meta-skill discovery is a particularly important direction for functional plasticity.

Information gain offers one possible learning signal, but reducing uncertainty alone does not establish the future usefulness of acquired knowledge. Other learning objectives like causal modeling, controllability, and action-conditioned predictions connect learning more directly and thus should be further explored.

The deeper problem is discovering which learning procedures remain effective across diverse worlds, and how those procedures should be internalized into model parameters.

To the best of my knowledge, so far, the community has no answers. Maybe some neo-labs are exploring this direction and may have early evidence of these design principles.

5. A discussion on such models’ commercialization embarrassment

These native continual learning models, if we finally figure out how to train them, will face an embarrassing commercialization dilemma when compared to competing models, the existing foundation models like GPT Astra, Claude Fable and many others.

• A general-purpose agent must have a moderate success rate in zero-shot learning when it first encounters an unfamiliar task. It should then improve efficiently from a limited amount of experience.

Therefore, we should consider two complementary capabilities:

• Initial performance on unseen tasks: A pretrained model arrives with sufficiently broad knowledge and learning skills to handle new tasks at an acceptable success rate.

• Continual improvement: Subsequent experience improves performance and learning efficiency, with gains persisting across sessions and tasks.

A model that needs thousands of downstream examples to become useful creates substantial deployment costs. It will backfire when shipped to customers!

These considerations also raise practical questions about memory persistence, interference, and how personalized continual learning can coexist with shared foundation model inference.

For native continual learning models, there can be a solution to the above dilemma.

TBC…

References