MAKAZHANALPAMYS / SOUP
Layer streaming is beta
Fine-tune LLMs from one YAML

The base model
never has to
fit.

Soup fine-tunes and post-trains LLMs in one command. The interesting part is layer streaming, which keeps the frozen base out of VRAM and feeds it to the GPU one decoder layer at a time.

That turns the question from how much model fits in your card into how much of one layer fits, which is a far smaller number.

$ pip install "soup-cli[train]"
$ soup init --template chat
$ soup train
Read the trainer github.com/MakazhanAlpamys/Soup
RTX 3050 Laptop, 4 GBFigures from the README
Resident base in VRAM does not fit
Llama-3.1-8B frozen base
4.00 GB CARD LIMIT
Layer streaming, one layer at a time 3.32 GB peak
3.32 GB peak
4.00 GB CARD LIMIT
Decoder layers, streamed
ON DISKIN VRAM NOWON DISK
Lane widths are illustrative. The measured figures are the README numbers: 119.6 tok/s at 3.32 GB peak on a 4 GB RTX 3050 Laptop, bit-exact against a resident run, reproduced on an H100 at 113.00 tok/s in the same 3.32 GB. The README notes the tok/s figure was taken on v0.72.2, before a v0.73.0 correctness repair, and has not been re-run on a 4 GB card since.
8B
Model trained
4GB
Laptop GPU
3.32GB
Peak VRAM
119.6
Tokens per second
One config

No SSH, no config hell

The README states the whole workflow is one config and one command, which is the part that decides whether a fine-tune actually gets attempted.

Bit-exact

Same result, less memory

Streaming only counts if the output matches. The project reports the streamed run is bit-exact against a normal resident run.

Opt-in

stream_layers: true

The feature is off by default and marked beta, so the default path stays the one people already trust.