Guide for onboarding new model architectures into NeMo AutoModel, including architecture discovery, implementation patterns, registration, and validation.
nemo-automodel-model-onboarding is a guide for adding support for new model architectures to NeMo AutoModel. It solves the problem of correctly integrating a Hugging Face model (LLM, VLM, or MoE) into the framework by walking through five phases in order: classify the architecture from config.json, name the exact implementation files under components/models/<name>/, identify registry and optional custom-config updates, and add the validation tests required before full checkpoint use.
The skill provides compact, prescriptive answer patterns for common cases. For a dense causal LM it specifies classification rules (a ForCausalLM architecture with no expert fields), the files to create (model.py, state_dict_adapter.py, __init__.py, optional config.py), the registry update in _transformers/registry.py, and the required tiny-config and layer-equivalence tests. For MoE models it insists on mapping router tensors separately, preserving routed-expert index order, mapping gate/up/down projections and shared experts explicitly, and adding key-map and numerical-equivalence tests rather than relying on from_pretrained or silent reshapes. For VLMs it checks vision_config, text_config, a ForConditionalGeneration architecture, the text backbone, vision tower, projector, processor assumptions, and image-text tests. Bundled pattern references (llm-patterns.md, vlm-patterns.md, moe-patterns.md, capabilities-and-precision.md) supply concrete implementation code such as combined QKV attention mixins, SwiGLU MLP layouts, and decoder-layer structure.
Target users are ML engineers extending NeMo AutoModel with new architectures. The skill enforces a clear routing boundary, deferring training-recipe YAML and launcher configuration to other skills. This is a read-and-write code-authoring reference for model integration with a strong correctness-and-testing emphasis; it does not run destructive operations or handle credentials.
Dense causal LLMs, Mixture-of-Experts (MoE) models, and vision-language models (VLMs), each with its own classification rules, implementation files, and required tests.
Classify the architecture from config.json, name the implementation files under components/models/<name>/, identify registry and optional custom-config updates, and add the validation tests needed before relying on full checkpoints.
MoE loading is error-prone, so the skill requires mapping router tensors separately from expert tensors, preserving routed-expert index order (never sorting, dropping, merging, or silently reshaping), mapping gate/up/down and shared experts explicitly, and adding key-map plus numerical-equivalence tests.
Classify as a VLM only when vision_config, text_config, and a ForConditionalGeneration architecture are all present, then check the text backbone, vision tower, projector, processor assumptions, and add tiny image-text tests.
Standalone training-recipe YAML questions about optimizers, datasets, schedulers, or trainer wiring (unless part of onboarding a new architecture), and launcher configuration. Those belong to the recipe-development and launcher-config skills.
Quick Setup:
.claude/skills/Repository
nvidia/skills