Model optimization library
Fixed ends.
Composable middle.
A library of state-of-the-art techniques — quantization, pruning, Neural Architecture Search, distillation, speculative decoding and sparsity. Python APIs compose them and export an optimized quantized checkpoint that downstream inference frameworks already load.
Signal path
Input
accepted, fixed
- Hugging Face
- PyTorch
- ONNX
Optimize
patchable, your call
- 01Quantization
- 02Pruning
- 03Neural Architecture Search
- 04Distillation
- 05Speculative decoding
- 06Sparsity
Compose them in any order through Python APIs. Techniques that require training run through Megatron-Bridge, Megatron-LM or Hugging Face Accelerate.
Export
deployment, fixed
- SGLang
- TensorRT-LLM
- TensorRT
- vLLM
Open source · pip install -U nvidia-modelopt[all]
Why fix the ends
The formats you can bring and the runtimes you can ship to are enumerable. Publishing them is what makes the middle safe to rearrange.
Why keep the middle open
No single technique wins everywhere. Composition is the product — quantize, then prune, then distil, in whatever order the model tolerates.
What crosses the gap
A checkpoint, not a service. Nothing downstream has to change to accept an optimized model.