ai-ml-security

An AI/ML security playbook covering, from an attacker's perspective, the ways machine-learning systems can be compromised, intended for security assessments of ML pipelines, model artifacts, and deployed models. It opens with model supply-chain risk, emphasizing that Python's pickle executes arbitrary code on deserialization and that PyTorch .pt/.pth files use pickle by default, making a downloaded model a code-execution vector. A format-risk table ranks .pt, .pkl, .joblib, and pickle-enabled .npy as dangerous while flagging .safetensors and .onnx as safe, and it details Hugging Face model poisoning (backdoored weights, malicious tokenizer configs, trust_remote_code) and dependency confusion in ML requirements.

Beyond the supply chain, it surveys adversarial examples with a taxonomy by attacker knowledge (white-box, transfer, query-based black-box, physical-world) and specific methods, namely FGSM, PGD, and Carlini and Wagner, plus physical-world attacks like adversarial patches, glasses, and stop-sign stickers. Model poisoning coverage includes training-data poisoning with backdoor triggers, label-flipping strategies, and gradient manipulation in federated learning, along with corresponding defenses such as robust aggregation (Krum, trimmed mean, median) and differential privacy. Model stealing is treated through query-based extraction, side-channel leakage (timing, confidence scores, top-K probabilities, power), and knowledge distillation from a black-box teacher.

Use this when threat-modeling or testing systems that load third-party models, expose inference APIs, train on external data, or run autonomous agents. It also covers data-privacy attacks (membership inference, model inversion, and gradient leakage) and LLM-specific and agent security concerns, and routes to companion skills for LLM prompt injection, insecure deserialization, and dependency confusion for deeper treatment of those specific areas.