Configure NeMo AutoModel job launches for interactive runs, Slurm clusters, and SkyPilot cloud execution.
nemo-automodel-launcher-config is a focused reference for configuring how NeMo AutoModel training and fine-tuning jobs are launched. It covers three launch methods: interactive (torchrun on the current node for single-node development), Slurm (batch submission to HPC clusters), and SkyPilot (cloud-agnostic submission to AWS, GCP, Azure, Lambda, or Kubernetes, including spot instances). It solves the problem of quickly answering launch-mechanics questions with the correct YAML block and expected runtime behavior, without requiring the agent to inspect the repository.
The skill provides compact answer patterns for common questions: a Slurm multi-node block with fields like job_name, nodes, ntasks_per_node, time, account/partition, container_image, hf_home, extra_mounts, env_vars, and master_port (noting the launcher derives WORLD_SIZE and sets MASTER_ADDR/MASTER_PORT); a SkyPilot spot block with cloud, accelerators, num_nodes, use_spot, disk_size, region, setup, and env_vars (with warnings that spot instances can be preempted and guidance to checkpoint frequently and resume via restore_from.path); and Nsight Systems profiling on Slurm (nsys_enabled wrapping the training command with nsys profile to produce a .nsys-rep report, treated as diagnostic-only because of overhead). It draws a clear routing boundary: it handles launch mechanics only, not model-architecture onboarding, which belongs to a separate skill.
Target users are developers and engineers configuring NeMo AutoModel job submissions across interactive, Slurm, and SkyPilot environments. Credential handling in the examples follows standard practice, referencing tokens like HF_TOKEN and WANDB_API_KEY through environment-variable indirection rather than embedding secret values. Its pre-publication benchmark returned an overall PASS verdict with high security and correctness scores.
Three: interactive (torchrun, default, single-node dev), Slurm (HPC batch submission with multi-node setup), and SkyPilot (cloud-agnostic submission supporting spot instances).
With a slurm: YAML block including job_name, nodes, ntasks_per_node, time, account or partition, container_image, hf_home, and optional extra_mounts and env_vars. The launcher derives WORLD_SIZE as nodes times ntasks_per_node and sets MASTER_ADDR and MASTER_PORT.
Spot instances can be preempted, so the skill advises setting a short checkpoint interval and resuming from a saved checkpoint path via restore_from.path.
Setting slurm.nsys_enabled: true wraps the training command with nsys profile and produces a .nsys-rep report in the Slurm job working/output directory. It is diagnostic-only and should be disabled for normal production training due to overhead.
Implementing or registering new model architectures, Hugging Face state-dict adapters, model files, or capability flags. Those model-onboarding tasks belong to the separate nemo-automodel-model-onboarding skill.
Quick Setup:
.claude/skills/Repository
nvidia/skills