tao-finetune-cosmos-reason

This NVIDIA skill drives supervised fine-tuning (SFT) and evaluation of Cosmos Reason video question-answering models, with the packaged default base model nvidia/Cosmos3-Nano. It solves the complexity of configuring a multi-GPU video-QA training run: it standardizes how the base model is sourced from HuggingFace, how FSDP parallelism is expressed (dp_shard_size for GPU count and dp_replicate_size for node count instead of the usual num_gpus/num_nodes), and when a checkpoint conversion to a Qwen3-VL HF safetensors directory is needed because some Cosmos-RL images cannot load the native Cosmos3 Omni format directly.

The skill is schema-driven and AutoML-aware. Generated TAO Core dataclass schemas live in schemas/<action>.schema.json with a manifest, and each emits a spec_template_<action>.yaml. Reference guides cover launch intake and preflight, evaluation (task types, LoRA eval, selective download, results), and AutoML/HPO policy including mapping objectives like accuracy targets to evaluation metrics and search spaces over learning rate, batch size, and weight decay. Train requests default to AutoML-on and route through a shared tao-run-automl runner when schemas and templates are packaged, falling back to direct training when AutoML is disabled or schemas are missing. Non-train actions (evaluate, inference, quantize) stay in this skill. It requires Docker with the NVIDIA container toolkit, and credentials are handled through the standard HF_TOKEN environment variable, with the user directed to accept the model license and provide a read token for gated models.

Target users are ML engineers and researchers fine-tuning or evaluating Cosmos Reason / Cosmos3-Nano video-QA models. Use cases include SFT and LoRA fine-tuning, accuracy-driven hyperparameter search, and evaluation of video-QA reasoning models.