RAFT: Why Your Time Series Model Needs a Memory

Time series forecasting has been dominated by increasingly complex architectures. From CNNs to RNNs, from Transformers to MLP-mixers, we've thrown everything at the problem of predicting the future from past data. Yet a fundamental question remains: Why are we forcing models to memorize patterns when we could simply look them up?

See RAFT (Retrieval-Augmented Forecasting of Time-series), a refreshingly simple yet powerful approach from researchers at KAIST, Max Planck Institute, and Google Cloud AI. Instead of relying solely on learned parameters, RAFT retrieves relevant historical patterns from the training data at inference time, and the results are striking.

What Is Retrieval-Augmented Time Series Forecasting?

Retrieval-Augmented Time Series Forecasting is an approach that improves prediction models by retrieving similar historical patterns at inference time rather than relying only on learned model parameters. Instead of forcing a neural network to memorize rare or unstable patterns within its weights, the system dynamically searches past data for comparable sequences and uses their subsequent values to guide future predictions. This retrieval mechanism acts as an external memory layer, helping the model respond more effectively to uncommon events or weak temporal correlations.

RAFT reframes time series forecasting by shifting the burden from memorization to retrieval. Rather than forcing models to internalize rare, unstable, or weakly correlated patterns within fixed parameters, it retrieves similar historical sequences at inference time and uses them to guide predictions. This simple yet powerful design improves performance across benchmarks, particularly in scenarios where traditional deep learning struggles. The key insight is clear: smarter access to historical data can outperform increasingly complex architectures.

The Problem: Memorization vs. Understanding

Real-world time series are messy. They exhibit: Non-stationary patterns that shift over time Rare events that appear infrequently but matter enormously Complex, non-deterministic processes lacking clear temporal correlations Traditional deep learning models must internalize all of this into their weights. As the authors note, "the advantages of indiscriminately memorizing all patterns, including noisy and uncorrelated ones, are questionable in terms of both generalizability and efficiency." Consider this scenario: Your model encounters a pattern it has seen only twice in training. Can it reliably predict what will come next? Probably not, unless it can retrieve those exact historical instances and use them as references.

The RAFT Solution: Retrieval as an Inductive Bias

Figure: RAFT retrieves similar historical patterns and uses their subsequent values to inform predictions. RAFT's architecture elegantly combines three components:

1.Multi-Period Downsampling

The model generates multiple views of the input series by downsampling at different periods (typically 1×, 2×, and 4×). This captures both short-term fluctuations and long-term trends; like how humans analyze charts at different zoom levels.

2.The Retrieval Module

For each period, RAFT:

• Extracts key patches from the historical training data using sliding windows
• Computes Pearson correlation between the current input and all historical keys (after removing offsets)
• Selects the top-m most similar patches and retrieves their corresponding value patches (what happened next historically)
• Aggregates these values using softmax-weighted attention

The similarity of metric matters: Pearson correlation specifically ignores scale variations, focusing purely on shape patterns, crucial for time series where amplitude may vary, but pattern structure remains predictive.

3.Lightweight Prediction Head

The retrieved information is projected, aggregated across periods, and concatenated with the processed input. A simple linear layer produces the final forecast. Key insight: The model doesn't need to learn rare patterns; it just needs to recognize when they reappear and fetch the appropriate historical continuation. Results: 86% Win Rate Across 10 Benchmarks

Average MSE across forecasting horizons (96, 192, 336, 720). Bold indicates the best performance. RAFT achieves state-of-the-art or second-best performance on 8 out of 10 datasets, with an average win ratio of 86% against contemporary baselines including TimeMixer, PatchTST, and DLinear. Notably, RAFT accomplishes this with a shallow MLP architecture, no Transformers, no complex attention mechanisms, no decomposition tricks. The retrieval module provides heavy lifting.

When Does Retrieval Help Most?

The authors conducted rigorous ablation studies using synthetic data to understand RAFT's strengths: Scenario 1: Rare Patterns Using autoregressive-generated short-term patterns with varying frequencies:

Lower MSE is better. Table adapted from the paper.  Finding: As patterns become rarer, retrieval provides greater benefit. The model simply cannot memorize what it rarely sees, but it can retrieve it.  Scenario 2: Temporally Uncorrelated Patterns  Replacing autoregressive patterns with random walks (where the next step depends only on noise, not history): 

When patterns lack temporal correlation, making it nearly impossible to learn through gradient descent, retrieval becomes even more critical. The improvement doubles compared to temporally correlated patterns. Beyond MLPs: Retrieval as a Universal Enhancer Can retrieval help existing architectures? The authors tested adding their retrieval module to AutoFormer, a Transformer-based model:

Consistent improvements across the board, suggesting retrieval augmentation could become a plug-and-play enhancement for time series models regardless of base architecture.

Practical Considerations

Computational Efficiency

Pre-computation: Key-value pairs are extracted once during training (O(N²) where N is series length) Training: 7.3 seconds per epoch on ETTm1 Inference: 1.9 seconds total Optimization: Increasing stride from 1 to 8 reduces pre-computation time by 89% with minimal performance degradation

Key Hyperparameters

Look-back window (L): Larger windows generally improve performance (consistent with linear model findings) Number of retrievals (m): 1, 5, 10, or 20 (dataset dependent) Temperature (τ): 0.1 provides good calibration of attention weights Periods: {1, 2, 4} captures multi-scale patterns effectively

The Broader Implication: RAG for Numbers

RAFT represents a philosophical shift in time series modeling, paralleling the Retrieval-Augmented Generation (RAG) revolution in NLP. Just as LLMs benefit from retrieving relevant documents rather than hallucinating facts, time series models benefit from retrieving relevant historical patterns rather than extrapolating limited learned representations. This approach addresses fundamental limitations: Catastrophic forgetting → Irrelevant, historical patterns remain accessible Distribution shift → Retrieval adapts to reappearing patterns without retraining Data efficiency → Rare patterns don't require massive datasets to learn

Looking Forward

The authors identify exciting directions for future work: Advanced similarity metrics: Beyond Pearson correlation, exploring learned similarity functions Selective retrieval: Determining when retrieval helps versus when learned patterns suffice Nonlinear dependencies: Capturing complex similarity measures for nonstationary characteristics External data integration: Retrieving from related time series or auxiliary features

Takeaways for Practitioners

Don't just build bigger models, consider what information should be learned vs. retrieved Rare patterns are retrievable, if your domain has infrequent but important events, retrieval augmentation is compelling Simplicity scales, RAFT's MLP backbone trains faster and infers cheaper than Transformer alternatives Retrieval is complementary, it can enhance existing models without architectural overhauls

Code & Resources

The official implementation is available at: https://github.com/archon159/RAFT For those working with time series in production, RAFT offers a pragmatic path forward: augment, don't just memorize. What patterns in your data might be too rare to learn but perfect for retrieval? About the Research: Sungwon Han, Seungeon Lee, Meeyoung Cha, Sercan Ö. Arik, and Jinsung Yoon. "Retrieval Augmented Time Series Forecasting." ICML 2025.