dm-limits-and-best-practices

A reference for applying Cognite Data Fusion Data Modeling (DMS) API best practices when writing or reviewing code that calls these APIs. It owns runtime reliability concerns such as concurrency limits, retries, throughput, and batching behavior, and defers graph-traversal payload correctness to a separate dm-graph-traversal skill. For exact, current limit numbers it points to the official Cognite documentation, while summarizing the key constraints to keep in mind: instance apply, delete, and query operations each have their own concurrent request limits, exceeding them returns HTTP 429, transformations consume much of the concurrency budget, instances.list has a max page size requiring pagination, instances.upsert accepts up to 1000 items per call, and in filters accept at most 1000 values per expression.

A significant section distinguishes search from filter. instances.search performs fuzzy free-text matching on string properties and supports an operator parameter, where AND (the default) narrows results so all terms must match and OR broadens them for exploratory or typeahead recall; search can also be combined with a filter for exact-match constraints. By contrast, instances.list and instances.query with a filter perform precise, deterministic matching (equals, range, in, hasData) with no fuzzy behavior. A decision table maps common needs to the right call: OR search for a search box, AND search to find a specific item by name, and filter on list or query for status, date-range, or enum constraints. For sets larger than 1000 IDs, a chunking pattern splits values into batches of 1000, issues requests in parallel through the task runner, and merges results.

The skill also documents a QueuedTaskRunner (semaphore) utility and advises always wrapping CDF API calls in a global cdfTaskRunner to bound concurrency and avoid 429s and deadlocks. If the project lacks a semaphore, it provides source for src/shared/utils/semaphore.ts: a generic QueuedTaskRunner class with a default of 15 concurrent tasks, a schedule method that queues tasks and supports a key option for deduplicating and aborting superseded tasks (throwing a provided AbortError), and an internal mechanism for consuming the next pending task as capacity frees up.