AI Tool ConsultantLLM Evaluation Skills for Claude
Report AI results you never have to retract.
Three skills from a local-LLM research project with 55+ auditable runs and zero retracted numbers: evaluation integrity rules, mechanical checks for LLM pipelines, and a fact-check that runs on any draft right before it's sent.
3 skills, each built from real work
LLM Evaluation Integrity
Re-derive every number twice, check confounds before crediting a change, compare on identical items, handle contamination, store complete outputs, keep metric definitions stable, and pick the cheapest decisive experiment.
- A 0.74 → 0.98 parse rate jump that wasn't the retry loop
- Same-item comparisons only
- Every rule tied to a real incident
LLM Output Validation
Schema validation with first-attempt and after-retry rates, invariant checks for judges and revisers, targeted repairs for catalogued failures, multi-stage extraction, and acceptance thresholds before writes.
- A judge model broke its one rule in 35 of 49 reviews
- First-attempt and after-retry rates logged separately
- Python patterns included
Fact-Check Before Sending
Extracts every checkable claim from a draft, verifies each against its source, reads linked pages, runs a sensitivity check on outbound content, and returns a verdict table plus minimal corrections.
- Built from real near-misses in emails to supervisors and stakeholders
- Reads linked pages, not just status codes
- Corrections keep the author's voice
Two minutes to install.
Before you buy
When can I buy?
Soon: checkout isn't open yet. Follow on LinkedIn to hear when it opens.
Which tools do the skills work with?
Claude.ai and Claude Desktop (upload a zip), Claude Code (plugin or folder), Codex (folder), and other agents that read the open SKILL.md format. A few skills include Python scripts; each skill page lists exactly what it needs.
How do I install them?
In Claude Code, add the private marketplace you're invited to after purchase and install the pack with one command. For Claude.ai and Claude Desktop, turn on code execution in Settings → Capabilities, then upload one zip per skill under Customize → Skills. For Codex, copy the skill folders into ~/.agents/skills/. Then ask normally; a skill switches on when your request matches it.
Do I get updates?
Yes. In Claude Code, turn on auto-update for the marketplace (/plugin → Marketplaces), or update a pack with claude plugin update <pack>@aitoolconsultant. Zip buyers can download the latest version from their purchase page.
Can I use them for my team or my clients?
Yes, for your own work and for client work you do. You can share a pack inside one team of up to five people at one company. Please don't resell or publish the files.
Do you see my data?
No. Skills are instructions and scripts that run inside your own AI tool. Nothing is sent to us.
Comparisons for this workflow
The skills work in Claude, Claude Code and Codex. If you're still picking the tool, start here.
Other workflows
LinkedIn Ghostwriting & Outreach
Write in a real person's voice, report what LinkedIn actually did, and run outbound that doesn't burn your domain.
SEO & Blog Content
Keyword research from evidence, one URL per keyword, and articles where every claim has a source.
Learn German with AI
The daily method behind months of measured German progress, as skills Claude can run for you.