digital-health-clinical-asr-eval

This skill is Stage 3 of NVIDIA's Clinical ASR Flywheel: a score-and-route stage that takes a NeMo-format manifest, transcribes it through a hosted NVIDIA ASR NIM, scores four metrics (WER/CER/KER/SER), and produces a five-section KER leaderboard with a by-ipa_source diagnostic. It solves the problem of objectively measuring clinical automatic-speech-recognition quality on domain-specific terminology, then deciding whether to advance to fine-tuning, loop back to data generation, or stop and harden the evaluation.

The workflow surfaces a clear data-transmission disclosure before any audio is sent: each manifest row's WAV plus its reference text goes to an external NVIDIA NVCF ASR service, while scoring runs locally in pure Python (or jiwer if installed). It strongly instructs users to pass only synthetic audio generated by Stage 2 and never real patient encounters or PHI. It defaults to the Parakeet TDT 0.6B v2 offline NIM with documented env-var overrides for model name, function ID, and self-hosted endpoints, and echoes the resolved function ID before spending API credits. A companion reference carries the full gRPC recipe and an alternate-NIM catalogue, and the skill routes off-topic requests (auth, model selection, deployment) to sibling skills. It requires an NVIDIA_API_KEY and treats its own SKILL.md as self-contained for methodology questions.

It targets healthcare ML engineers and TME teams building and evaluating clinical ASR models. Typical use cases include benchmarking a candidate ASR NIM against a curated clinical term list, generating a leaderboard to compare Parakeet versus Whisper, and gating progression through the flywheel.