Notes from the nest · 980 posts
Engineering notes written by the people building Crowkis. Comparisons, use cases, economics, internals, security, operations, and nothing written just to rank.
Observability dashboard + Prometheus, shows hit rate, saved spend, safety blocks, and memory pressure live, and exposes Prometheus /metrics, all in the box. Here's how Crowkis does it and why it matters for cost and safety.
For the questions where 'close enough' is unacceptable, pricing, legal, brand lines, CPIN serves a human-approved answer verbatim, records who approved it, and never lets the model improvise.
A cache lives in the hot path of every request. The language choice isn't aesthetic, it's the difference between predictable microseconds and mystery pauses.
CDOC adds documents with auto-chunking and metadata, then searches them with filters and optional reranking, no separate vector database.
Building SQL generation tools on the Mistral SDK? Add a semantic cache so the same schema questions and query shapes stop costing full price.
Building internal copilots on Ollama? Add a semantic cache so employees asking overlapping questions of the same knowledge base stop costing full price.
Building education tutors on Semantic Kernel? Add a semantic cache so students asking the same concepts thousands of times stop costing full price.
Building HR assistants on LangChain? Add a semantic cache so the same policy questions from every employee stop costing full price.
Redis-compatible (RESP3), speaks RESP3 so redis-py, ioredis, and Lettuce connect unmodified across 40+ commands, adoption is a port change. Here's how Crowkis does it and why it matters for cost and safety.
Pull, run, first hit in the dashboard, with no config file, no signup, and no environment variables you're required to set. We timed it. It holds.
Redis is magnificent infrastructure for exact-match workloads. LLM traffic isn't one. Here's why speaking the same protocol doesn't mean solving the same problem.
CSESSION stores a multi-turn conversation, reads the recent window, and semantically searches the whole thing, so 'as I mentioned earlier' works.
Empty strings, 100 KB values, null bytes, emoji, 16 threads hammering across tenants. The stress harness throws 84 nasty checks at Crowkis and counts the cross-tenant leaks. The leak count is zero.
Building devops copilots on the Mistral SDK? Add a semantic cache so the same runbook and incident questions stop costing full price.
Building chatbots on Ollama? Add a semantic cache so high-volume conversational traffic that repeats constantly stop costing full price.
Building multi-agent systems on Semantic Kernel? Add a semantic cache so a swarm of agents asking overlapping questions stop costing full price.
Building IT helpdesk bots on LangChain? Add a semantic cache so the same tickets and fixes, endlessly stop costing full price.
Pure-Rust LSM storage engine, is a from-scratch WAL + MemTable + SSTable + bloom-filter + compaction engine in Rust, no RocksDB and no FFI. Here's how Crowkis does it and why it matters for cost and safety.
Most caches only remember good answers. Crowkis also remembers bad ones, flag a hallucinated or harmful response once, and CCHECKBAD catches every paraphrase of the question that would have reproduced it.
Nowhere else do thousands of people ask the same fifty questions, all day, in every phrasing imaginable. Crowkis was practically designed in a support queue.