Architecture
culprit has three narrow seams:
GitRepositoryshells out to the realgitexecutable. Git remains the only version-history database and prompts remain plain files.EvalRunnersends a resolved prompt and golden cases through aProvider. Deterministic assertions run locally; semantic assertions use an explicit judge model.EvalCachestores response and assertion evidence in local SQLite using content-derived keys. Credentials are accepted only by provider SDKs and are never included in cache payloads.
Provider adapters implement one generate(system, user, model) method. OpenAI and Anthropic are built in. The OpenAI-compatible adapter makes local runtimes configurable without binding the core to one runtime. Contributors can also wrap an arbitrary Python callable.
Semantic diff first produces empirical before/after evidence. Its LLM summary receives that evidence and cannot silently accept or merge anything. Textual Git diff remains available as a fallback primitive rather than the headline result.