Llama 3.1 8B Instruct

Cheap enough to put inside a loop. Where classification and routing work usually ends up.

$argosy up --model llama-3-1-8b-instruct

ttft19ms

p50 from the nearest warm region

Throughput486tok/s

Single stream, no batching

Context128K

Full window, no sliding truncation

Price$0.06

Per million tokens served

Small, fast, and cheap enough to put in a loop. Most people who start on the 70B and then measure end up moving classification and routing work down to this one.

Deploymentdep_8f14e4

Parameters8B

Quantisationfp8

LicenceLlama 3.1 Community

Home regioniad1

Extra replicas11

p50 latency21ms

p99 latency52ms

Requests / s4100

Figures are illustrative — this is a demonstration site.

Last 30 minutesiad1

cache hitqueue 0ms errors 0

Also in the catalogue

LL Llama 3.3 70B Instruct 38ms $0.34 QW Qwen2.5 72B Instruct 41ms $0.36 QC Qwen2.5 Coder 32B 27ms $0.18 GM Gemma 2 27B 24ms $0.16

Put the model next to the user.

One command, thirty-one regions, and an invoice that matches what you actually served.

Deploy a model Talk to an engineer