Published research library
CoDA-GQA-L
Experiment with an attention cache that stays bounded.
A PyTorch attention module combining a recent-token window with exact and summary memory banks. Inspect fixed-size cache behavior on synthetic inputs before attempting model integration.
What you can do with it
Who it's for: Inference engineers and researchers studying memory-limited attention.
An attention-module smoke test that reports tokens processed and cache allocation.
Your first useful result
Install in your development environment
pip install coda-gqa-l- 01
Run the attention-only example
Copy the complete Quick start example from the repository. It creates CoDAGQALandmarkPerf2, prefills random embeddings, and then decodes more tokens.
- 02
Inspect bounded state
Compare the reported token position and cache size as the sequence grows. This tests the module mechanics; random inputs do not measure language quality.
- 03
Plan a model-level comparison
Use the documented two-phase adaptation procedure and compare against standard attention with the same model, data, and evaluation settings.
Know the limits
- Reported cache sizes depend on model dimensions, precision, batch size, and bank configuration.
- The repository documents quality loss and the need for bounded-cache training. The module is not a quality-preserving drop-in replacement for every pretrained model.