tao-finetune-huggingface-model
Fine-tune any HuggingFace CV / VLM / LLM model on local NVIDIA GPUs inside an NGC PyTorch container. Use when the user wants to fine-tune a HuggingFace model (full or LoRA), train a vision / VLM / LLM model end-to-end, generate a reproducible HF training pipeline, smoke-test a HuggingFace model locally before scale-up, push a fine-tuned model to the HF Hub with a model card, or emit a self-contained rerun skill for an existing HuggingFace finetune. Supports image classification, object detection
Security Assessment
About tao-finetune-huggingface-model
This NVIDIA skill orchestrates local fine-tuning of HuggingFace models on NVIDIA GPUs inside an NGC PyTorch container. It solves the problem of turning a HuggingFace model ID plus a dataset into a reproducible, deployable training pipeline without hand-assembling containers, scripts, and configs. It supports a broad task range — image classification, object detection, semantic/instance/panoptic segmentation, depth estimation, image-text-to-text VLM (SFT/LoRA), and LLM SFT/DPO/GRPO — with both full and LoRA fine-tuning.
The workflow is a fixed six-step process: inspect and qualify the model, pick hardware and the NGC image, research (live-fetching the model card, HF repo examples, task docs, and papers, with curated `references/*.md` as a fallback), generate scripts and smoke-test, train plus evaluate plus run inference, then push to the HF Hub with a model card and emit a self-contained rerun skill. It ships worked examples (ConvNeXt/CIFAR-10, DETR/CPPE-5, SegFormer/FoodSeg103, SmolVLM/VQAv2), extensive references for datasets, hardware, error playbooks, reporting, and testing, and an NVSkills-Eval BENCHMARK.md documenting a PASS verdict with 100% security scores. All settings live in `config.yaml` rather than hardcoded Python.
It targets ML engineers and researchers who want to fine-tune or smoke-test a HuggingFace model locally before scaling up, and to produce a reproducible, publishable pipeline. Credential handling is careful and defensive: `HF_TOKEN` is read only when a model/dataset is gated or when pushing to the Hub, and its value is never read — only its presence is checked; `WANDB_API_KEY`/`WANDB_PROJECT` apply only when Weights & Biases logging is enabled. It requires Docker with the NVIDIA Container Toolkit, an NVIDIA GPU (driver ≥545, ≥24 GB VRAM for models up to ~3B), and roughly 40 GB of free disk.
FAQ
What tasks and fine-tuning methods does it support?
Image classification, object detection, semantic/instance/panoptic segmentation, depth estimation, image-text-to-text VLM (SFT/LoRA), and LLM SFT/DPO/GRPO, with both full and LoRA fine-tuning.
What hardware and software are required?
Docker plus the NVIDIA Container Toolkit, an NVIDIA GPU (driver ≥545, at least 24 GB VRAM for models up to about 3B parameters), and roughly 40 GB of free disk. Training runs inside an NGC PyTorch container.
How are credentials handled?
`HF_TOKEN` is only needed when the model/dataset is gated or when pushing to the Hub, and its value is never read — the skill only checks presence via `[ -n "$HF_TOKEN" ]`. WANDB_API_KEY/WANDB_PROJECT are used only when WandB logging is enabled.
Can I reproduce or rerun a finetune later?
Yes. The final workflow step emits a self-contained rerun skill for an existing finetune, and all parameters live in `output_dir/config.yaml` rather than being hardcoded in Python, so runs are reproducible.
How do I supply data?
Provide exactly one of: a HuggingFace `dataset_id`, a `local_dataset_path` (with optional format among imagefolder, coco, voc, jsonl, arrow, parquet, csv), or omit it to let the agent recommend popular datasets.
All Files
73 filesInstall tao-finetune-huggingface-model
Quick Setup:
- Copy the skill folder to
.claude/skills/ - Claude will automatically detect and use the skill
Repository
nvidia/skills