google-agents-cli-eval

The `google-agents-cli-eval` skill is designed to help users systematically evaluate agents built using the Google Agent Development Kit (ADK). It addresses the need for structured assessment of agent performance by providing guidance on evaluation methodology, metrics, and iterative improvement processes. This skill allows users to run evaluation sets, debug evaluation scores, and compare results across different versions of an agent, ensuring that the agent behaves as expected in a variety of scenarios. By focusing on evaluation rather than deployment or scaffolding, it provides a targeted approach to improving agent quality and reliability within the ADK ecosystem.

Key features of this skill include support for running eval sets through the `agents-cli eval run` command, references for evaluation criteria, dynamic user simulation, and tool-specific trajectory scoring. It covers both standard and multimodal evaluation inputs, detailing schema requirements and metric limitations. Users can leverage the Eval-Fix loop methodology to iteratively diagnose low scores, adjust agent instructions or eval sets, and rerun evaluations to track improvements. Additional guidance is provided on common pitfalls, such as improper threshold tuning, flaky evals, or misalignment between expected outputs and agent behavior.

This skill is primarily intended for developers and testers working with ADK agents who need to ensure high-quality performance and correctness. Typical use cases include evaluating new agent versions, debugging failing evaluation cases, refining agent instructions, and tracking improvements across iterative fixes. It is suitable for both scaffolded projects and standalone agents where evaluation rigor is critical, making it valuable for teams aiming to systematically enhance agent capabilities and reduce unintended behavior in production scenarios.