Google has detailed a new review and validation system for its Agent Skills repository, which now contains more than 170 skills and has passed 21,000 GitHub stars.
The system adds automated checks for semantic versioning, evaluation suites, link validation and an AI-based reviewer before a human subject matter expert can approve a skill change.
Each pull request in Google's internal repository now passes through six automated gates before human approval. These cover structural conformance, skill linters, semantic versioning, evaluation suite validation, link checks and an AI review agent.
The approach reflects a broader shift in how Google treats agent instructions. Rather than handling a skill as static documentation, it treats each update as a product release because even small wording changes can alter how an AI agent reasons, selects tools or executes tasks.
Release discipline
A central part of the process is strict semantic versioning. Every skill carries a version number in its metadata, and any behavioural change to instructions, scripts or references requires a version bump before a pull request can proceed.
New skills must start at 1.0.0, while standard semantic versioning reset rules apply to major and minor changes. The version number is intended for developers and package tooling rather than for model use at runtime, allowing teams to compare local installations with the upstream catalogue and decide when to update.
Google also validates evaluation suites that measure agent performance with and without a given skill. These checks confirm that referenced prompt files exist and are not empty, that each expectation tests a single criterion, that duplicate prompts or conflicting expectations are flagged, and that every pull request links to a fresh evaluation report based on the latest commit.
AI reviewer
Alongside linters and schema checks, Google has introduced an AI review agent to assess instruction quality and design standards. Using a dedicated review skill, it inspects proposed changes and identifies problems that syntax checks do not catch.
One of its main tests is whether a task belongs in a natural-language instruction at all. Google says authors often fall into what it calls the deterministic trap, where rigid command-line sequences are written directly into Markdown instructions even though they are better placed in executable scripts.
Under the review rules, deterministic operations such as command-line sequences, parsers and deployment steps should sit in scripts, while Markdown instructions should focus on work that requires judgement, such as diagnosis, architecture choices or synthesis. The AI reviewer also checks that skills follow a three-level structure: concise metadata, a compact main instruction file and separate supporting reference documents.
Google's reviewer enforces a 500-line limit for the main SKILL.md body and blocks chained references that force agents to follow multiple document hops. It also flags interactive helper scripts, requiring scripts to accept command-line flags or environment variables, provide help documentation, and separate structured output from error messages.
Repository growth
The repository has grown from 15,000 to 21,000 GitHub stars since the earlier phase of the project, according to Google. It now hosts skills from teams across the company and distributes them across multiple AI platforms.
Google argues that this scale has made manual review less effective. Unvetted edits can introduce subtle bugs, while broken links and weak evaluation design can erode trust in the repository over time.
To address link decay, Google validates relative links before merge and runs scheduled external link checks across the public repository. Broken links are then reported to skill owners for remediation.
Lessons learned
Google also outlined several lessons from six months of repository history. One was the need to scope CI checks carefully after early rules blocked minor typo fixes while failing to inspect some newly added skills in deeper subdirectories.
Another was the importance of atomic expectations in evaluation suites. Compound expectations produced inconsistent ratings from LLM-based judging systems, while breaking them into single checks made results clearer and easier to debug.
Google also found that documentation drift sets in quickly in a multi-contributor repository. To counter this, it now relies on generated files and CI checks to keep core indexes up to date rather than asking contributors to maintain them manually.
Cost control was another factor in the pipeline's design. Fast deterministic checks run first and stop the process if they fail, so heavier LLM-based reviews and evaluations do not run unnecessarily on broken changes.
This ordering is intended to reduce token use and CI runtime as contributors make frequent small edits during review, while preserving stricter checks for changes that pass the initial gates.
Human approval remains the final step, typically handled by the skill's subject matter expert and, for new skills, a member of Google's Central Skills team.