built-in-metrics

built-in-metrics is a LaunchDarkly skill for instrumenting an existing codebase with AI config metric tracking, so an application's provider calls report duration, input/output tokens, success/error, and time-to-first-token to LaunchDarkly's Monitoring tab. Its guiding principle is to add the least ceremony that still captures the needed metrics by choosing the highest applicable tier of a four-tier ladder rather than dropping straight to manual tracker calls.

The four tiers are: a managed runner (create_model/createModel returning a ManagedModel whose run() captures everything automatically) for chat-style calls; a provider package plus trackMetricsOf using the package's getAIMetricsFromResponse extractor for non-chat shapes; a custom extractor with trackMetricsOf when no provider package exists (Anthropic direct, Gemini, Cohere, custom HTTP); and raw manual tracking (trackDuration, trackTokens, trackSuccess/trackError, trackTimeToFirstToken) for streaming and unusual shapes. The skill emphasizes that every provider uses the same generic wrapper shape and that only the extractor changes per provider. It ships extensive reference files with concrete Python and Node patterns for OpenAI, Bedrock, Gemini, Anthropic, LangChain, Strands, streaming, and the metrics API, including guidance on error handling (trackMetricsOf records and re-throws) and TTFT tracking for streams.

It targets backend engineers using the LaunchDarkly server-side AI SDKs (Python launchdarkly-server-sdk-ai or Node @launchdarkly/server-sdk-ai, version 0.20.0+) who already have a config and want observability over their LLM calls. It is a defensive/observability skill that instruments existing code with official SDK APIs.