Notes from the nest · 980 posts
Engineering notes written by the people building Crowkis. Comparisons, use cases, economics, internals, security, operations, and nothing written just to rank.
Vision queries are expensive and repetitive, the same product photo, the same screenshot, asked about again and again. Crowkis caches image-plus-text lookups so a repeated visual question is a hit.
CROWKIS_MEMORY_LIMIT means what it says, no GC mood swings, no mystery RSS, eviction that engages before the kernel has opinions.
Cost pressure pushes teams toward cheaper, dumber models. Caching offers the opposite trade: keep frontier quality, pay small-model prices on the traffic that repeats.
Building sales enablement tools on LangChain.js? Add a semantic cache so reps asking the same product questions stop costing full price.
Building SQL generation tools on Ollama? Add a semantic cache so the same schema questions and query shapes stop costing full price.
Building internal copilots on DSPy? Add a semantic cache so employees asking overlapping questions of the same knowledge base stop costing full price.
Building education tutors on LangGraph? Add a semantic cache so students asking the same concepts thousands of times stop costing full price.
Add a semantic cache to Semantic Kernel so repeated and reworded questions are served for free, no rewrite, self-hosted.
No phone-home, offline license verification, one binary. The deployment story for networks that treat outbound packets as incidents.
Every DIY semantic cache is a vector database, a Redis, a cron job, and a prayer. Crowkis is the version where the parts were designed for each other.
Building knowledge base assistants on LangChain.js? Add a semantic cache so the same lookups across a team all day stop costing full price.
Building devops copilots on Ollama? Add a semantic cache so the same runbook and incident questions stop costing full price.
Building chatbots on DSPy? Add a semantic cache so high-volume conversational traffic that repeats constantly stop costing full price.
Building multi-agent systems on LangGraph? Add a semantic cache so a swarm of agents asking overlapping questions stop costing full price.
Durable, per-user memory for Semantic Kernel agents that survives restarts and consolidates contradictions, self-hosted, zero egress.
Chain-of-thought tokens are the most expensive ones you buy. Crowkis extracts the thought's skeleton, abstracts the specifics, and recomposes it for the next input that shares its shape.
Clinical-adjacent assistants repeat administrative and informational answers constantly, but every cached byte is regulated. This is what compliance-mode caching looks like.
Every sane checklist says don't write your own storage engine. We did it anyway. Here's the actual reasoning, the architecture, and the parts that were painful.
Building research assistants on LangChain.js? Add a semantic cache so overlapping literature and summary questions stop costing full price.
Building API documentation bots on Ollama? Add a semantic cache so the same endpoint questions from every developer stop costing full price.