Research and Development|Live

Promptctl

CLI for prompt engineering: version control, test-driven evaluation, A/B testing with statistical confidence.

Overview

Prompts as Code. Markdown + YAML frontmatter, JSON test fixtures, bulk evaluation with pass/fail. A/B testing with 95% confidence intervals via bootstrapped sampling. Local React dashboard for traces and cost analysis.

Specs

FormatMarkdown with YAML frontmatter, Git-tracked

TestingExact match, regex, semantic similarity, JSON schema

A/B Testing95% confidence intervals via bootstrapped sampling

ProvidersGoogle Gemini, OpenAI, Anthropic, Ollama

Features

Prompt versioning with Markdown + YAML frontmatter
Four assertion types: exact match, regex, semantic similarity, JSON schema
Parallel execution across providers
A/B testing with 95% confidence intervals
Local React dashboard for history and cost analysis

Tech Stack

TypeScriptNode.jsReactViteExpressGoogle GenAI

Interested in Promptctl?

Let's discuss how this can work for you.

Get in Touch