harvard-artifacts-collection-etl-analytics

harvard-artifacts-collection-etl-analytics is a hands-on skill for building ETL pipelines and analytics dashboards from the Harvard Art Museums API using Python, SQL, and Streamlit. It closely parallels the companion Harvard artifacts skill, addressing the same goal of turning a public museum API into a queryable database and interactive dashboard, and describes the architecture as API to ETL to SQL to Analytics to Visualization.

The skill supplies runnable Python: a fetch function that pages the API (filtering to artifacts with images) with rate limiting, extraction functions that split artifacts into metadata, media, and color DataFrames, and a loading layer that creates a MySQL/TiDB schema via mysql-connector-python. It documents obtaining a free Harvard API key and storing configuration in environment variables (API key and database host, user, password, name) exported in the shell, which is standard credential-setup guidance. It highlights 20+ analytical queries and auto-generated visualizations with Plotly.

It targets data engineers, analysts, and learners who want a cultural-heritage-data reference implementation for pagination, JSON-to-relational transformation, schema design, and dashboarding. Common use cases include practicing ETL construction, modeling nested API responses into normalized tables, and building a Streamlit analytics front end. Like its sibling, it directs users to clone an upstream GitHub repository for the complete application, so it operates partly as a guided entry point to that external project.