guidesJune 6, 2026· 5 min read

A/B testing prompts in production with CPROMPT

Version a prompt, split traffic across versions with sticky per-user bucketing, render variables, and roll back — without a deploy or a feature-flag service.

Prompts are production logic, and CPROMPT lets you treat them like it: versioned, A/B-split, and rollback-able. The experiment is sticky per user, so a given subject always sees the same variant instead of flickering per request.

version, split, render

from crowkis import CrowkisClient
c = CrowkisClient(host="127.0.0.1", port=6379)

# two versions of a support prompt
c.cprompt_set("support_reply", "You are terse. Answer: {{q}}")        # v1
c.cprompt_set("support_reply", "You are warm and concise. Answer: {{q}}")  # v2

# 50/50 weighted split across the two versions
c.cprompt_abset("support_reply", [["1", 50], ["2", 50]])

# sticky per user — same subject, same variant, every time
version = c.cprompt_ab("support_reply", subject=user_id)
prompt = c.cprompt_render("support_reply", version=version, vars={"q": question})

Every write creates a new version, so the prompt you shipped last week is still retrievable. If v2 underperforms, `cprompt_rollback` reverts it; the whole thing survives restarts, so a reschedule doesn't reset your experiment.

In plain words: Store prompts like versioned code, split traffic between versions per user, and roll back a bad one — without a deploy or a separate feature-flag tool.