How to cache the Gemini SDK LLM calls with Crowkis
Add a semantic cache to the Gemini SDK so repeated and reworded questions are served for free, no rewrite, self-hosted.
Production LLM traffic is deeply repetitive, and repetition is exactly what a bill is made of. If you build with the Gemini SDK, most of that repetition is invisible in your code but very visible on your bill. A semantic cache in front of your model calls fixes it.
The lowest-friction path is the OpenAI-compatible gateway: point the Gemini SDK's base URL at Crowkis and every model call flows through a semantic cache. Repeated and reworded prompts are served from cache with no upstream call; new ones pass through and get cached.
# point the Gemini SDK at the Crowkis gateway base_url = "http://127.0.0.1:6380/v1" # semantic cache in front of your provider
On repetitive workloads this cuts LLM costs up to 60-70% on repetitive workloads, and every hit comes back with a confidence score so reuse stays safe. Drop it in over RESP, gRPC, REST, or MCP, no rewrite required.