AirLLM · huge models, small GPUssecond take
One layerat a time.
AirLLM only ever keeps one layer on the GPU at a time, so the VRAM you need depends on the model’s layer size — not its total size. No quantization, distillation or pruning required.
$ pip install airllm
Parameter labels scale with model sizeVRAM, as the README reports it
27BQwen3.8-27B
dense VL
dense VL
70BLlama 3.x 70B
full precision
full precision
235BQwen3-235B
MoE
MoE
405BLlama 3.1
405B
405B
671BDeepSeek-V3
2.8TKimi K3
A hundred times more parametersabout the same card
Same one line
Pass a Hugging Face repo ID and go bigger with the exact same call: AutoModel.from_pretrained("deepseek-ai/DeepSeek-V3").
The weights move to disk
The original model is first decomposed and saved layer-wise, so the cache needs the disk space. The README is plain about it: the bottleneck is mainly disk loading.
Training, too
Frozen base weights stream from disk one decoder layer at a time while only the adapters stay resident — Qwen3.8-27B trains in ~2GB at sequence length 512.