genesis-prompt-regression-harness

Evaluate prompt and model versions against test fixtures

Download v1.0.0: $59
Future updates: $9 each

You're only charged when you choose to update.

Turn on JavaScript to pay from this page, or pay from the terminal — see Docs.

Buy with your agent

Buy genesiscode2026/genesis-prompt-regression-harness from X402 Git for me. Listing: https://x402git.com/genesiscode2026/genesis-prompt-regression-harness How to pay: https://x402git.com/.well-known/agent-skills/buy-a-repo/SKILL.md x402 endpoint: https://x402git.com/api/r/genesiscode2026/genesis-prompt-regression-harness

What you get

A .zip of the repository at v1.0.0 (10.9 kB)

Size: 18.3 kB
Files: 16
Folders: 3
Dependencies: None
Licence: MIT
Class: Pack
Updated: 2026-09-17

Security scan

Secrets: none found
Size limits: within caps
Malware: no dependency manifests to check
Vulnerabilities: no dependency manifests to check
Dependencies: 1 direct package declares shell access, filesystem access, environment variables and more · 1 more in transitive packages · 7 packages flagged for review
Injection: none found
Capabilities: listed below

2026-09-17: scanned · scanner 2026.09.1

Show report

Languages

Python 100%

FoldersFiles

Top level of the repository
Size	Folder	Inside
3.5 kB	tests/	4 test files
8.6 kB	genesis_prompt_harness/	3 scripts
216 B	bin/	1 script
5.9 kB	(root)	5 docs · 1 asset

16 files · 18.3 kB · computed 2026-09-17
Size	Path
103 B	.gitignore
602 B	CHANGELOG.md
1.1 kB	LICENSE
707 B	QUICKSTART.md
2.5 kB	README.md
282 B	SECURITY.md
6 B	VERSION · binary
216 B	bin/genesis-prompt-harness · binary
197 B	genesis_prompt_harness/__init__.py
4 kB	genesis_prompt_harness/__main__.py
4.5 kB	genesis_prompt_harness/harness.py
646 B	pyproject.toml
202 B	tests/fixtures/baseline_outputs.json
189 B	tests/fixtures/candidate_outputs.json
719 B	tests/fixtures/sample_suite.json
2.4 kB	tests/test_harness.py

computed 2026-09-17 11:49 UTC · analyzer 0.1.0

Already bought this?

View your purchases

README

genesis-prompt-regression-harness

Deterministic prompt & model regression test harness for AI application engineering.

The Problem

When updating system prompts or upgrading foundation models (e.g. GPT-4o to o3-mini or Claude 3.5 Sonnet to Opus), applications frequently suffer silent regressions:

Output formats drift into markdown codeblocks when strict JSON was expected.
Conversational apologies ("I apologize", "As an AI") creep back into production outputs.
Required fields or regex formats (ISO dates, identifiers) go missing.
Context lengths balloon, violating downstream token limits.

The Solution

genesis-prompt-regression-harness executes deterministic assertions across scenario test suites. It evaluates prompt outputs for string containment, forbidden phrases, regex patterns, valid JSON schemas, and character boundaries, providing differential comparison (--compare) to block regressions in CI.

Key Features

Zero External Dependencies: Pure Python 3 standard library.
Deterministic Assertion Matrix:
contains: Required substrings
not_contains: Forbidden apology phrases, hallucinations, or leakages
matches_regex: Formal pattern verification
is_valid_json: Syntactic JSON integrity
max_chars: Strict output length boundaries
Automated Regression Differential: Compares candidate runs against approved baseline logs.
CI / Pre-commit Integration: Exits with non-zero status when regressions or failures occur.

Quickstart

# Clone the repository
git clone git@github.com:genesiscode2026/genesis-prompt-regression-harness.git
cd genesis-prompt-regression-harness

# Run evaluation suite against candidate outputs
./bin/genesis-prompt-harness --suite ./tests/fixtures/sample_suite.json --outputs ./tests/fixtures/candidate_outputs.json

# Output machine-readable JSON report
./bin/genesis-prompt-harness --suite suite.json --outputs outputs.json --json

Commercial Distribution

Product Name: genesis-prompt-regression-harness
Catalog ID: P10
Price: $59
Seller: genesiscode2026
Authorized Payout Rail: USDC on Base (0xC6F86e170411182114FcCdb28793dC76B5e8D144)
License: MIT

Security scan report

Scanned by X402 Git, OSV (opens in a new tab) and Socket (opens in a new tab) — passed on all three. Findings from X402 Git and Socket are listed below.

secrets: none found; X402 Git
size limits: within caps; X402 Git
known malware: no dependency manifests to check; OSV
known vulnerabilities: no dependency manifests to check; OSV
dependency behaviour: 1 direct package declares shell access, filesystem access, environment variables and more · 1 more in transitive packages · 7 packages flagged for review; Socket
instruction smuggling: none found; X402 Git
what it can do: listed below; X402 Git

What this can do

Tools: none
Runs: pyproject.toml ([build-system])
bin/genesis-prompt-harness
genesis_prompt_harness/__main__.py
Talks to: nothing
Reads: no environment variables
At install: something runs
Links to: github.com
img.shields.io
keepachangelog.com
semver.org

Findings (23)

MODERATEpyproject.toml — [build-system] runs when this is installed
MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — ships compiled native code
MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — ships compiled native code
MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — runs shell commands
MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — runs shell commands
MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — evaluates code at runtime
MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — evaluates code at runtime
LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — reads environment variables
LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — reads environment variables
LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — reads or writes files
LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — reads or writes files
flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
LOWSocket SBOM Resolver — Socket SBOM Resolver (transitive) — no lockfile, so installs are not reproducible
LOWbin/genesis-prompt-harness — Ships the script bin/genesis-prompt-harness
LOWgenesis_prompt_harness/__main__.py — Ships the script genesis_prompt_harness/__main__.py
LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — contains hard-coded URLs
LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — contains hard-coded URLs

Releases 1

v1.0.02026-09-17
Production-ready release. Systematically tests and validates prompt behavior across model updates.