The throughput ceiling we won't hide — and the fix
On v0.2.2, throwing 16 threads at Crowkis got the same throughput as one. That's a real ceiling, we found it in our own harness, and here's both why it happened and how the embedding-deferral work lifts it.
Here's a benchmark result most vendors would quietly drop: on v0.2.2, single-threaded semantic throughput was about 9 operations per second, and sixteen threads delivered… about 9 operations per second. A scaling factor of 1.0. We're publishing it because the reason is instructive and the fix is real.
Two bottlenecks stacked: a single-writer actor and a synchronous embed on the hot path.
Two things stacked up. First, Crowkis funnels cache decisions through a single-writer actor — a deliberate design that makes correctness and crash-recovery provable, but means writes don't run in parallel. Second, and dominant here, every semantic op ran the ONNX embedding inline, so each thread spent ~115 ms in the same synchronous model call. Sixteen threads waiting on the same serial embed is sixteen threads in a line.
The fix is to get embedding off the hot path: cache embeddings for repeats and defer the rest so the actor isn't blocked on the model. With that work, exact-repeat throughput jumps from ~51 ops/sec to ~3,550 ops/sec single-threaded — a 70× improvement — and the multi-thread number finally moves above the floor.
70× on the repeat path. The single-writer actor remains the next ceiling to lift.
A benchmark you only run to win isn't a benchmark, it's an ad. We run ours to find the ceiling — then we go raise it.
Concurrent reads against the single-writer actor are the next infrastructure step, and we're honest that distributed throughput is not where Crowkis competes today. For the workload it's built for — agent fan-out and chatbot traffic, where the same handful of questions repeat constantly — the repeat path is the one that matters, and that's the one the deferral work transforms.