bigquery-pipeline-audit
Audits Python + BigQuery pipelines for cost safety, idempotency, and production readiness. Returns a structured report with exact patch locations.
Security Assessment
About bigquery-pipeline-audit
The bigquery-pipeline-audit skill is designed to audit Python and BigQuery pipelines, focusing on critical aspects such as cost safety, idempotency, and production readiness. By analyzing the codebase, this skill helps data engineers identify potential runaway costs, ensure that reruns do not corrupt data, and make failures visible. The structured report generated by the skill highlights exact patch locations, allowing for targeted fixes rather than extensive rewrites, thus streamlining the auditing process.
FAQ
How does the bigquery-pipeline-audit skill identify cost exposure?
It locates every BigQuery job trigger and external call, assessing their placement in loops or retry blocks, and checks for configurations like maximum bytes billed.
What are the requirements for using this skill?
The skill requires a Python environment with access to BigQuery and the ability to analyze Python scripts that interact with BigQuery.
Can this skill be used for pipelines not written in Python?
No, this skill is specifically designed for Python + BigQuery pipelines.
What happens if the script runs one BQ query per date or entity in a loop?
The skill will hard fail, indicating a critical design flaw that needs to be addressed.
Is there a way to ensure that reruns do not create duplicate data?
Yes, the skill checks for deduplication logic in write operations and suggests best practices for safe writes.
Install bigquery-pipeline-audit
Quick Setup:
- Copy the skill folder to
.claude/skills/ - Claude will automatically detect and use the skill
Repository
github/awesome-copilot