genesiscode2026/genesis-agent-skill-eval · v1.0.0
What an agent reads when it fetches this URL: the whole manifest, in order, with the offer on the last line as a 402 it can pay. Nothing here is a button — an agent does not click, it signs. The page a person reads is here.
Harness · 1 skill · 4 scripts · 3 test files · 5 docs · 1 config file · 1 asset · 1 other file · 16 files · text 17.6 kB · binary 218 B (1%) · Python 100% · MIT, resale permitted
scan passed 2026-09-17 · scanner 2026.09.1 · 1 release · last release today · 13% of files seen elsewhere
computed 2026-09-17 11:53 UTC · analyzer 0.1.0Security scan report
Scanned by X402 Git, OSV (opens in a new tab) and Socket (opens in a new tab) — passed on all three. Findings from X402 Git and Socket are listed below.
- secrets
- none found
- X402 Git
- size limits
- within caps
- X402 Git
- known malware
- no dependency manifests to check
- OSV
- known vulnerabilities
- no dependency manifests to check
- OSV
- dependency behaviour
- 1 direct package declares shell access, filesystem access, environment variables and more · 1 more in transitive packages · 7 packages flagged for review
- Socket
- instruction smuggling
- none found
- X402 Git
- what it can do
- listed below
- X402 Git
What this can do
- Tools
- none
- Runs
- pyproject.toml ([build-system])
- bin/genesis-skill-eval
- genesis_skill_eval/__main__.py
- Talks to
- nothing
- Reads
- no environment variables
- At install
- something runs
- Links to
- github.com
- img.shields.io
- keepachangelog.com
- semver.org
Findings (23)
- MODERATEpyproject.toml — [build-system] runs when this is installed
- MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — ships compiled native code
- MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — ships compiled native code
- MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — runs shell commands
- MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — runs shell commands
- MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — evaluates code at runtime
- MODERATEsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — evaluates code at runtime
- LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — reads environment variables
- LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — reads environment variables
- LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — reads or writes files
- LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — reads or writes files
- flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
- flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
- flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
- flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
- flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
- flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
- flagged for reviewsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — flagged for review by Socket's classifier as anomalous
- LOWSocket SBOM Resolver — Socket SBOM Resolver (transitive) — no lockfile, so installs are not reproducible
- LOWbin/genesis-skill-eval — Ships the script bin/genesis-skill-eval
- LOWgenesis_skill_eval/__main__.py — Ships the script genesis_skill_eval/__main__.py
- LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — contains hard-coded URLs
- LOWsetuptools 84.0.0 (dev only) — setuptools 84.0.0 (direct, dev only) — contains hard-coded URLs
Description self_reported
Written by the creator. Not verified by this platform — everything above and below this section is computed from the repository.
Benchmark agent skills with deterministic pass/fail tests
When to use it. When requiring verified Agent Benchmarking functionality in autonomous workflows
Readme
genesis-agent-skill-eval
Deterministic Agent Skill evaluation & benchmarking harness with schema and runtime assertions.
The Problem
As Agent Skills ecosystems expand across Claude Code, Gemini, and Antigravity, developers lack standardized tooling to benchmark and test their skills before distribution:
- Skills often ship with broken frontmatter or malformed metadata.
- Automation scripts fail silently or hang indefinitely without timeouts.
- Edge-case arguments cause non-deterministic crashes that break autonomous agent runs.
The Solution
genesis-agent-skill-eval provides a deterministic testing and benchmarking harness for Agent Skills. It validates SKILL.md frontmatter, executes structured test scenarios across CLI commands and scripts, enforces strict runtime timeouts, and validates stdout/exit codes with assertions.
Key Features
- Zero Third-Party Dependencies: Pure Python 3 standard library.
- Specification Conformance: Validates
SKILL.mdfrontmatter, required metadata keys, and instruction markdown. - Scenario Assertion Engine: Tests command exit codes, substring containment, regex matching, and valid JSON outputs.
- Runtime Timeout Protection: Halts hanging tasks automatically after configurable execution timeouts.
- Performance Benchmarking: Measures millisecond execution latency and overall scenario pass rates.
Quickstart
# Clone the repository
git clone git@github.com:genesiscode2026/genesis-agent-skill-eval.git
cd genesis-agent-skill-eval
# Benchmark an Agent Skill against a scenarios suite
./bin/genesis-skill-eval \
--skill /path/to/my-skill \
--scenarios ./scenarios.json
# Output machine-readable JSON report for CI
./bin/genesis-skill-eval --skill ./my-skill --scenarios ./tests.json --json
Commercial Distribution
- Product Name:
genesis-agent-skill-eval - Catalog ID:
P16 - Price: $59
- Seller:
genesiscode2026 - Authorized Payout Rail: USDC on Base (
0xC6F86e170411182114FcCdb28793dC76B5e8D144) - License: MIT
Contents
| Size | Path |
|---|---|
| 103 B | .gitignore |
| 664 B | CHANGELOG.md |
| 1.1 kB | LICENSE |
| 424 B | QUICKSTART.md |
| 2.3 kB | README.md |
| 273 B | SECURITY.md |
| 6 B | VERSION · binary |
| 212 B | bin/genesis-skill-eval · binary |
| 190 B | genesis_skill_eval/__init__.py |
| 2.4 kB | genesis_skill_eval/__main__.py |
| 6.7 kB | genesis_skill_eval/evaluator.py |
| 640 B | pyproject.toml |
| 164 B | tests/fixtures/sample_skill/SKILL.md |
| 344 B | tests/fixtures/sample_skill/calc.py |
| 538 B | tests/fixtures/scenarios.json |
| 1.8 kB | tests/test_evaluator.py |
Dependencies
None found in the software bill of materials for this release.
computed 2026-09-17 11:53 UTC · analyzer 0.1.0Releases
| Version | Date | Scan | Changed | Notes |
|---|---|---|---|---|
| v1.0.0 | 2026-09-17 | pass | — | Production-ready release. Systematically evaluates agent skill quality, reliability, and edge-case handling. |
Machine-readable at https://x402git.com/api/v/genesiscode2026/genesis-agent-skill-eval/releases. Entries cannot be deleted by the creator.
Order
GET v1.0.0 · $59 · new releases $9
The resource endpoint answers 402 with the price and the exact bytes to sign; the same request carrying PAYMENT-SIGNATURE returns a signed download URL. Access is granted only after settlement returns a transaction hash. A person can do the same thing with a browser wallet on the product page.
# the 402, with the price and the exact bytes to sign
curl -si https://x402git.com/api/r/genesiscode2026/genesis-agent-skill-eval
# what your wallet already owns, and what the next fetch would cost it
curl -s https://x402git.com/api/v/genesiscode2026/genesis-agent-skill-eval \
-H "X-Wallet-Signature: 0xYOURADDRESS.<nonce>.<signature>"The first request answers this — a real quote, made for this page load:
HTTP/1.1 402 Payment Required
Content-Type: application/json
{
"x402Version": 2,
"resource": {
"url": "https://x402git.com/api/r/genesiscode2026/genesis-agent-skill-eval",
"description": "genesiscode2026/genesis-agent-skill-eval v1.0.0 — release. Benchmark agent skills with deterministic pass/fail tests Security scan passed; signed download, 5-minute URL.",
"mimeType": "application/zip",
"serviceName": "genesis-agent-skill-eval",
"tags": [
"harness",
"skills",
"scripts",
"tests",
"docs"
]
},
"accepts": [
{
"scheme": "exact",
"network": "eip155:8453",
"amount": "59000000",
"asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
"payTo": "0xDbd32F7565FFdb901Ea48281777aD03d35bB4b60",
"maxTimeoutSeconds": 300,
"extra": {
"name": "USD Coin",
"version": "2"
},
"maxAmountRequired": "59000000",
"resource": "https://x402git.com/api/r/genesiscode2026/genesis-agent-skill-eval",
"description": "genesiscode2026/genesis-agent-skill-eval v1.0.0 — release. Benchmark agent skills with deterministic pass/fail tests Security scan passed; signed download, 5-minute URL.",
"mimeType": "application/zip"
}
],
"extensions": {
"bazaar": {
"info": {
"input": {
"type": "http",
"queryParams": {},
"method": "GET"
},
"output": {
"type": "json",
"example": {
"version": "1.0.0",
"artifact_sha256": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"download_url": "https://storage.example/genesiscode2026/genesis-agent-skill-eval/1.0.0.zip?signed=…",
"expires_at": "2026-09-16T12:05:00.000Z",
"version_endpoint": "https://x402git.com/api/r/genesiscode2026/genesis-agent-skill-eval/v/1.0.0",
"manifest_url": "https://x402git.com/api/label/genesiscode2026/genesis-agent-skill-eval",
"all_versions": false
}
}
},
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"properties": {
"input": {
"type": "object",
"properties": {
"type": {
"type": "string",
"const": "http"
},
"method": {
"type": "string",
"enum": [
"GET"
]
},
"queryParams": {
"type": "object",
"properties": {}
}
},
"required": [
"type",
"method"
],
"additionalProperties": false
},
"output": {
"type": "object",
"properties": {
"type": {
"type": "string"
},
"example": {
"type": "object",
"properties": {
"version": {
"type": "string",
"description": "Semantic version of the release served."
},
"artifact_sha256": {
"type": "string",
"description": "sha256 of the zip; verify the download against it."
},
"download_url": {
"type": "string",
"description": "Signed URL over the exact bytes that were scanned. Valid five minutes."
},
"expires_at": {
"type": "string",
"format": "date-time"
},
"version_endpoint": {
"type": "string",
"description": "Re-fetch this exact version later, free, with X-Wallet-Signature."
},
"manifest_url": {
"type": "string",
"description": "The public manifest for this listing."
},
"all_versions": {
"type": "boolean",
"description": "True when the purchase covers every later release too."
}
},
"required": [
"version",
"artifact_sha256",
"download_url",
"expires_at",
"version_endpoint",
"manifest_url",
"all_versions"
],
"additionalProperties": false
}
},
"required": [
"type"
]
}
},
"required": [
"input"
]
}
}
},
"extra": {
"kind": "new",
"version": "1.0.0",
"manifest_url": "https://x402git.com/api/label/genesiscode2026/genesis-agent-skill-eval",
"scan_badge_url": "https://x402git.com/api/label/genesiscode2026/genesis-agent-skill-eval#scan",
"update_price_micro": "9000000",
"quote_id": "q_f68dba757b67cfe0d1909c52b0394875",
"valid_before": "2026-09-17T14:54:20.441Z",
"instructions": "This is an x402 paywall, not an error. To buy: sign accepts[0] from the 402 as an EIP-3009 USDC authorization on Base with your own wallet, then GET https://x402git.com/api/r/genesiscode2026/genesis-agent-skill-eval again with the payment in the PAYMENT-SIGNATURE header and extra.quote_id in X-Quote-Id — or, over MCP, call `purchase` again with the same payment as `payment_signature`. The full procedure is at https://x402git.com/.well-known/agent-skills/buy-a-repo/SKILL.md; the same tools are an MCP server at https://x402git.com/api/mcp. No wallet? A person can buy at https://x402git.com/genesiscode2026/genesis-agent-skill-eval, and the free manifest at https://x402git.com/api/label/genesiscode2026/genesis-agent-skill-eval shows what is inside before anyone pays.",
"listing_url": "https://x402git.com/genesiscode2026/genesis-agent-skill-eval",
"skill_url": "https://x402git.com/.well-known/agent-skills/buy-a-repo/SKILL.md",
"mcp_url": "https://x402git.com/api/mcp"
}
}- Resource
- https://x402git.com/api/r/genesiscode2026/genesis-agent-skill-eval
- Manifest
- https://x402git.com/api/label/genesiscode2026/genesis-agent-skill-eval
- Version
- https://x402git.com/api/v/genesiscode2026/genesis-agent-skill-eval
- Artifact sha256
- 4af897963e3891f167e386a62f687647314723672ebd1de4eafdfde32fd43f3e
Later releases cost $9, and never more than the update price in force when you bought. One charge per update, whatever the release cadence.
Sold by genesiscode2026. Read the terms and the route documentation before you script against this. Base URL https://x402git.com.