NewWhite paper, Draft 3: a cost model and system design for dense AI workloads. Read it →
dense infra

Coming soon · Canestack Labs

Thousands of AI agents.
One machine.

Agents, sandboxes and small models sit idle most of the time. Dense infra starts each one in milliseconds, holds almost nothing while it waits, and wakes it on the next request. You pay for the work, not for the waiting.

console.denseinfra.com/overviewDesign preview
OverviewFleetCostWake latencyNew sandboxes
Sandboxes1,240on 6 hosts
Awake now66 5.3%peak 141
Memory held71 GiBkept warm: 412 GiB
Wake p95148 mstarget 250 ms
Per sandbox$0.38$2.21 kept warm
Awake sandboxes and memory held 24 h
kept warm: 412 GiB
Where sandboxes are by sleep tier
Awake6641.2 GiB
Paused21422.6 GiB
Reclaimed7027.1 GiB
Snapshotted2580 GiB
aws-use1-aHealthy
148 / 16012 awake131 ms p95
c7g.8xlarge · 64 GiB
gcp-euw4-aHealthy
155 / 1608 awake164 ms p95
n2-highmem-16 · 128 GiB
rack-1Wakes slow
47 / 1603 awake212 ms p95
Own server · 96 GiB

The dense infra console, an early design. Not built yet; the numbers follow the white paper's model.

Start in ms

No warm pools. A sandbox starts when its request arrives.

Sleep at zero

Idle sandboxes give their memory back and wait on disk.

Your cloud

Runs in your own account, on your servers or on your laptop.

01The problem

Agents wait. The bill doesn't.

A coding agent runs a test, then waits a minute for the model. A support bot answers one message every few minutes. Starts are slow, so teams keep every sandbox warm and pay for each idle second.

Kept warm 6 of 100 working

Dense 6 of 100 working

  1. 01

    Warm pools

    You pay for sandboxes that do nothing.

    Agents are busy about one second in twenty. A warm pool bills all twenty.

  2. 02

    Cold starts

    Starting fresh takes seconds.

    Containers and VMs that boot on demand make the user wait, so nobody lets them sleep.

  3. 03

    Someone else's cloud

    Hosted sandboxes take your code elsewhere.

    And your credits and commitments go unspent while you pay a second bill.

02The platform

Wake, sleep, pack and run anywhere.

Four jobs, one runtime. Each is a known hard problem; we are building them together, on the runtime under Kiri.

01

Wake on the request

A front door holds every sandbox's sockets. When a request arrives for one that is asleep, it holds the connection, restores the sandbox and passes the request in. The caller sees a slow first byte, never an error.

How the door works →
agent-0142 · wake 148 ms
  1. 0 msRequest arrives at the doorHTTP POST /run
  2. 2 msSandbox found in a snapshoton local disk
  3. 37 msMemory mapped, pages load as touchedlazy restore
  4. 141 msProcess running againsame PID, same state
  5. 148 msFirst byte to the callertarget 250 ms
02

Sleep in tiers

Idle sandboxes step down: paused in memory, then pages compressed, then a snapshot on disk. Each step waits until sleeping saves more than waking costs, so the busy ones never notice.

The break-even rule →
Sleep policy · default applies to 1,240 sandboxes
Awakerunning a request—
Pausedwakes in ~2 msafter 30 s idle
Reclaimedwakes in ~40 msafter 5 min
Snapshottedwakes in ~150 msafter break-even, 9 min
341 GiB handed back to the hosts in the last hour
03

Pack the host

Identical images, libraries and model weights are loaded once and shared. Memory goes to the sandboxes that are working, so one machine holds thousands instead of dozens.

How we will measure it →
0 awake of 1,240 on one hostIllustration
04

Run in your cloud

Connect a cloud account and dense infra starts hosts in it, or joins machines you already run: a Kubernetes node pool or your own servers. Your code stays in your account and your credits get spent.

Questions about clouds →
Clouds this month
AWSus-east-1 · 3 hosts712 sandboxes
Google Cloudeurope-west4 · 2 hosts436 sandboxes
Your servers1 rack92 sandboxes
+Connect a cloudAzure, Oracle, DigitalOcean, Kubernetes…

Runs where you already pay

  • AWS
  • Google Cloud
  • Azure
  • Oracle Cloud
  • DigitalOcean
  • Akamai
  • Vultr
  • Hetzner
  • Kubernetes
  • Your servers

A first look

What it could feel like.

Early design for the first piece: many agent sandboxes on your own machine, in Kiri. Not built yet; names, commands and screens will change.

TerminalDesign preview
$ kiri sandbox run --count 100 --image agent:latest --sleep-when-idle
✓ 100 sandboxes from 1 shared image
  awake 6   asleep 94   memory held only by the awake

$ kiri sandbox ls
NAMESTATEIMAGEDOINGagent-001awakeagent:latestrunning testsagent-002asleepagent:latestwaiting on the modelagent-003asleepagent:latestwaiting on the modelagent-004wakingagent:latestrequest arrivedagent-005asleepagent:latestidleagent-006awakeagent:latestwriting a patch… 94 more

$ kiri sandbox freeze agent-001
✓ agent-001 frozen · memory returned to the host · wakes on the next request
$ 

Illustration of a design that isn't built. Counts are examples, not measurements.

03Where we start

Kiri already starts a container in 22 ms.

Kiri, our container runtime, replaces the dockerd, containerd, shim and runc chain with one Rust daemon and one small VM. Its start and packing numbers are what dense infra builds on.

22msA full docker run, start to finish.Docker Desktop 178 ms
Colima 152 ms
3.1msA container lifecycle through the Docker API.Colima 88 ms
Docker Desktop 163 ms
88.9/sContainers started in parallel, per second.Colima 12.9/s
Docker Desktop 11.4/s
19.7/sContainers packed per second in the density test.Colima 4.1/s
Docker Desktop 3.2/s

One kiri-bench run, one Apple Silicon Mac, every engine limited to 2 CPUs and 8 GB, 2 November 2026. Laptop numbers: servers at thousands of workloads per host are the work ahead. Benchmarks and method.

05How we will prove it

The first honest density benchmark.

Every "thousands per machine" figure we could find comes from the company selling it. Ours is a test anyone can rerun on equal hardware, against Docker, Firecracker, gVisor and a Kubernetes node, with the tests we lose left in.

METRIC 1

Workloads per host

How many fit before p95 latency moves.

METRIC 2

Wake time

Request to first byte, from asleep.

METRIC 3

Idle cost

Memory and CPU held while doing nothing.

METRIC 4

Cost per useful second

The machine's cost over the seconds of real work.

Be first to run dense.

Dense infra is not available yet. Join the waitlist and we will write when the first piece, many agents on your own machine, is ready to try.