Modern recommendation systems face a paradox:
- Users expect instant personalization — adaptation after a single click.
- But recommender models rely on slow retraining cycles — hourly or nightly.
- Transformers and deep models can store long-term knowledge, but cannot update that knowledge during inference.
The result, as the recent Nested Learning (NeurIPS 2025 https://abehrouz.github.io/files/NL.pdf) paper from Google Research argues, is that today’s models suffer from a kind of anterograde amnesia:
They cannot integrate new experiences (interactions) into long-term memory while running.
Nested Learning proposes a biologically inspired framework for fixing this: multiple memory systems operating at different time scales, all inside the model.
This article will explain:
- The core ideas from the Nested Learning paper
- How its equations (13/16, 17/20, 25) formalize fast and slow learning
- How we can apply this pattern to retail recommendation using a simple Python implementation
Then apply the concept in product recommendation as
What Is Nested Learning?
The paper argues that a neural network is not one learner, instead; it is several learners stacked inside each other, each running at a different speed:
Three components align perfectly with deep learning:
Fast Memory: Attention
This is Eq. 13/16 in the paper.
Middle Memory: Optimizer Momentum
Optimizers like SGD+momentum or Adam actually maintain an internal state: This is Eq. 17–20. Momentum is not a hack; it is a memory of recent surprise signals.
Slow Memory: Model Weights
The slow weights update with in Eq. 25.
Equation Walkthrough
Let’s walk through the three equations that matter most.
(A) Fast Associative Memory (Eq. 13/16)
The paper shows that attention is itself a gradient descent update on an internal memory matrix:
Why outer product?
- kₜ is the key
- vₜ is the value
- Their outer product is a short-lived associative trace
This lets the model adapt instantly to new behavior — without retraining.
(B) Momentum as Memory (Eq. 17/20)
Momentum stores a compressed gradient history:
This becomes the optimizer-level fast memory, storing “what surprised me recently.” This plays a critical role in slow weight updates.
(C) Outer Slow Weight Update (Eq. 25)
At much slower intervals, the real model parameters update:
The key insight: Slow weights do NOT update every step; they update using the compressed memory from the fast system. This gives us fast online adaptation + slow long-term learning.
Turning Nested Learning into a Personalized Recommender System
Most personalized recommenders today behave like Transformers:
- They infer & score interactions online based on customer’s preference and interaction
- But the model does not adapt or modify their parameters online
- Updates require large batch retraining
Let’s walk through what actually happens when a customer opens your e-commerce site. Imagine a customer lands on e-commerce homepage. In most recommenders today, The customer receives a static ranking from inferencing and scoring based on:
- the customer’s long-term profile
- historical and in-session behavior
- users who look like the customer
Nothing in a traditional model adapts in real time to the customer’s behavior as he browses. Nested Learning changes that. In our implementation, every action the customer takes immediately reshapes the model’s fast memory, updating his recommendations as he clicks and scrolls. Over time, these fast-memory insights are gradually consolidated into the slow weights, becoming part of the model’s long-term knowledge.
STEP 1 — Customer views an Item → Fast Memory Updates (Eq. 13 / 16)
He clicks on a yogurt product. NL model forms the feature vector:
x = concat(user_embedding[Emily], item_embedding[Yogurt])
Then NL computes attention projections:
k = x @ W_k
v = x @ W_v
q = x @ W_q
Now Eq. 13 / 16 kicks in:
In the code:
self.memory += np.outer(v, k)
This does something very important in adaptation: The recommender creates a short-term associative memory — A lightweight “trace” linking Emily’s current preferences to yogurt-type items. This memory could be session-scoped or multi-channel scoped: it reflects what the customer is interested in right now, not since last time model was fully trained. We don’t need to retrain anything yet — the fast memory handles this live.
STEP 2 — Customer clicks or ignores items → Momentum Memory Updates (Eq. 17 / 20)
This customer now clicks a second item — a protein bar. Or maybe he ignores it.
NL system computes the prediction error:
error = prob - label
grad = error * x
This gradient feeds into the momentum memory:
self.state.grad_memory =
beta * self.state.grad_memory +
(1 - beta) * grad
This memory captures: the direction customer’s preferences are shifting
- If he clicks on dairy items → gradient moves weights toward dairy signals
- If he ignores spicy snacks → gradients move weights away from those signals
Noise is filtered out. Trends are kept. This is mid-term memory — not immediate like attention, not slow like retraining.
STEP 3 — Fast Weights Update Instantly
Fast weights adjust right away using the momentum memory:
self.state.fast_weights =
self.state.fast_weights - eta_fast * self.state.grad_memory
As every new interactions changes the customer’s personalized ranking. Within just 3–5 interactions, the recommender has customized itself for Emily in ways that traditional systems cannot — it is like a “personalized recommendation model” for the customer.
STEP 4 — Offline (Slow) Update Consolidates Across All Users (Eq. 25)
When it’s time to retrain the full model, we can consolidate the fast weights into the slow weights using Eq. 28 and Eq. 29. These updates incorporate not only the accumulated fast weights but also the input feature vectors, ensuring that long-term memory becomes feature-aware rather than just a stored average.
In today’s industry models:
- Personalization is driven by offline retraining
- Online adaptation is limited to bandits or heuristic rules
- The core model cannot update itself during a session
Nested Learning changes the architecture fundamentally:
Real-time “Personalized Model”
Every interaction reshapes the fast memory and fast weights. NL recommender adapts on the fly like a human store associate:
“Ah, this customer picked Greek yogurt and protein bars — let me adapt my memory to learn more about protein-rich snacks.”
Momentum memory handles short-term preference drift
If the customer suddenly shifts from “healthy snacks” to “baking supplies,”
the model updates seamlessly.
Slow weights learn long-term, population-level behavior
The long term memory system still captures:
- lifetime events
- loyalty habits
- household profiles
- seasonal trends
- slow shift of purchase patterns
The dual-memory system is biologically plausible
Humans use:
- short-term working memory (fast attention)
- medium-term consolidation
- long-term memory
And Nested Learning mirrors this architecture.
NOTE: This is not a production recommender but a conceptual implementation that mirrors the Nested Learning architecture.
"""Recommender with fast online updates and periodic slow commits."""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Dict, Iterable, Tuple
import numpy as np
from numpy.typing import ArrayLike, NDArray
@dataclass
class OnlineOfflineState:
"""Tracks slow and fast weights alongside momentum memory."""
slow_weights: NDArray[np.float64]
fast_weights: NDArray[np.float64]
grad_memory: NDArray[np.float64]
def _sigmoid(x: NDArray[np.float64]) -> NDArray[np.float64]:
return 1.0 / (1.0 + np.exp(-x))
@dataclass
class OnlineOfflineRecommender:
"""Nested learning recommender with fast online and slow offline updates.
Online step:
- Eq. (12)-(16): update user-specific fast attention memory.
- Eq. (17)-(20): update momentum-style gradient memory.
- Fast weights use the smoothed gradient immediately for scoring.
Offline step (less frequent):
- Eq. (25): apply the slow gradient descent step to slow weights using
the accumulated gradient memory, then resynchronise fast weights.
"""
num_users: int
num_items: int
user_dim: int = 8
item_dim: int = 8
memory_dim: int = 8
eta_fast: float = 0.05
eta_slow: float = 0.01
beta: float = 0.9
slow_update_interval: int = 10
rng: np.random.Generator = field(default_factory=lambda: np.random.default_rng(0))
state: OnlineOfflineState = field(init=False)
W_k: NDArray[np.float64] = field(init=False)
W_v: NDArray[np.float64] = field(init=False)
W_q: NDArray[np.float64] = field(init=False)
W_fast_out: NDArray[np.float64] = field(init=False)
memory: NDArray[np.float64] = field(init=False)
user_embeddings: NDArray[np.float64] = field(init=False)
item_embeddings: NDArray[np.float64] = field(init=False)
step_count: int = field(init=False, default=0)
def __post_init__(self) -> None:
input_dim = self.user_dim + self.item_dim
slow_weights = self.rng.normal(scale=0.01, size=input_dim)
fast_weights = slow_weights.copy()
grad_memory = np.zeros_like(slow_weights)
self.state = OnlineOfflineState(
slow_weights=slow_weights,
fast_weights=fast_weights,
grad_memory=grad_memory,
)
self.W_k = self.rng.normal(scale=0.05, size=(input_dim, self.memory_dim))
self.W_v = self.rng.normal(scale=0.05, size=(input_dim, self.memory_dim))
self.W_q = self.rng.normal(scale=0.05, size=(input_dim, self.memory_dim))
self.W_fast_out = self.rng.normal(scale=0.05, size=self.memory_dim)
self.memory = np.zeros((self.memory_dim, self.memory_dim), dtype=np.float64)
self.user_embeddings = self.rng.normal(scale=0.1, size=(self.num_users, self.user_dim))
self.item_embeddings = self.rng.normal(scale=0.1, size=(self.num_items, self.item_dim))
@property
def input_dim(self) -> int:
return self.user_dim + self.item_dim
def _features(self, user_id: int, item_id: int) -> NDArray[np.float64]:
return np.concatenate(
[self.user_embeddings[user_id], self.item_embeddings[item_id]],
axis=0,
)
def _attention(self, x: NDArray[np.float64], *, update: bool) -> Dict[str, NDArray[np.float64]]:
k = x @ self.W_k
v = x @ self.W_v
q = x @ self.W_q
if update:
self.memory += np.outer(v, k)
y_fast_vec = self.memory @ q
fast_scalar = float(self.W_fast_out @ y_fast_vec)
return {"k": k, "v": v, "q": q, "fast_vec": y_fast_vec, "fast_scalar": fast_scalar}
def score(self, user_id: int, item_id: int) -> float:
x = self._features(user_id, item_id)
attention = self._attention(x, update=False)
y_slow = float(self.state.fast_weights @ x)
logit = y_slow + attention["fast_scalar"]
return float(_sigmoid(np.array([logit]))[0])
def recommend(self, user_id: int, item_ids: NDArray[np.int_]) -> NDArray[np.float64]:
return np.asarray([self.score(user_id, item) for item in item_ids], dtype=np.float64)
def online_step(self, user_id: int, item_id: int, label: float) -> Dict[str, float | NDArray[np.float64]]:
x = self._features(user_id, item_id)
attention = self._attention(x, update=True)
y_fast = float(self.state.fast_weights @ x)
logit = y_fast + attention["fast_scalar"]
prob = float(_sigmoid(np.array([logit]))[0])
loss = -label * np.log(prob + 1e-9) - (1 - label) * np.log(1 - prob + 1e-9)
error = prob - label
grad = error * x
self.state.grad_memory = self.beta * self.state.grad_memory + (1 - self.beta) * grad
self.state.fast_weights = self.state.fast_weights - self.eta_fast * self.state.grad_memory
self.step_count += 1
committed = False
if self.step_count % self.slow_update_interval == 0:
self.offline_update()
committed = True
return {
"prob": np.array([prob]),
"loss": np.array([loss]),
"error": np.array([error]),
"grad": grad,
"grad_memory": self.state.grad_memory.copy(),
"fast_weights": self.state.fast_weights.copy(),
"slow_weights": self.state.slow_weights.copy(),
"attention_memory": self.memory.copy(),
"attention_fast_scalar": np.array([attention["fast_scalar"]]),
"committed": np.array([1.0 if committed else 0.0]),
}
def offline_update(self):
"""
Apply the correct offline slow update following Eq. (28)-(29).
W_{t+1} = W_t (I - x x^T) - eta_slow * (u_t ⊗ x_t)
"""
# Pull slow weights into variable
W = self.state.slow_weights
# gradient memory stores the EMA of gradient: g_t = error * x_t
# This corresponds to ∇_y L ⊗ x_t (outer product)
grad_W = self.state.grad_memory # shape: (input_dim,)
# --- Build contraction matrix (I - x xᵀ) ---
# Need the last input x_t used online; store x_t in state during online_step
x_t = self.state.last_x # shape: (input_dim,)
I = np.eye(self.input_dim)
contraction = I - np.outer(x_t, x_t)
# --- Apply Eq. (28)/(29) ---
# W_{t+1} = W_t (I - x_t x_t^T) - η * (grad ⊗ x_t)
# Since W is a row vector here, simplify the multiplication
W_new = W @ contraction - self.eta_slow * grad_W
# Update slow and fast weights (fast resets to slow snapshot)
self.state.slow_weights = W_new
self.state.fast_weights = W_new.copy()
def online_train(self, interactions: Iterable[Tuple[int, int, float]]) -> None:
for user_id, item_id, label in interactions:
self.online_step(user_id, item_id, label)
Why This Matters for Product Recommendation
Product recommendation has several realities:
- Users behave differently across sessions
- Their taste can shift in minutes
- New items appear constantly
- Traditional retraining cycles cannot keep up
Nested Learning offers a practical recipe for hybrid online + offline learning:
- Fast weights: adapt immediately after every session engagement per user
- Slow weights: consolidate knowledge across users and time
- Momentum memory: smooths noise and prevents overfitting to single clicks
- Attention memory: captures short-term context like “what the user is focusing on right now”
This mirrors how humans learn: something surprising modifies short-term memory quickly, but only repeated exposure consolidates it.