Model Memory: Parameters × Bytes per Parameter (FP16, INT8, INT4)
All lessons
- Model Memory: Parameters × Bytes per Parameter (FP16, INT8, INT4)Free
- KV-Cache Memory per Token & Max Concurrent Sequences Pro
- Throughput vs Batch Size: Tokens per Second per GPU and per User (Chart) Pro
- Latency: Time to First Token, Time per Token & Total Response Time Pro
- GPUs for the Peak: Little's Law, Throughput & ROUNDUP Pro
- Utilisation & Cost per 1M Tokens from the GPU Hourly Price Pro
- Self-Hosting vs API: Monthly Cost & the Break-Even Volume (Chart) Pro
- Capacity Plan: Six Months of Growth, Headroom, GPUs & Budget Pro
You are not signed in — the lab works, but your progress is not saved. Start free
Loading the spreadsheet…