Developer Community

If you have tests that assume the a specific result ordering in instance list or query results without specifying a sort order when querying for recently upserted nodes, you may in the future see that APIs can return results in a different order than they were upserted, and possibly see your tests fail.The change in ordering happens when we insert data in the DM graph, and is required to make sure we correctly serialise operations and avoid deadlocks when processing upsert operations. The previous implementation is prone to rare race conditions that, while not having been a frequent problem before, may become a problem now as we work to optimize DM API performance and add support for more sophisticated access control in instance upsert operations.While this change is observable if your test hard codes an assumption that items are returned in the same order that they were inserted, this behavior was never specified, let alone guaranteed. The API behavior remains in line with previously

Dedicated CDF environment for AI agent & app developmentWe've been working through how to give AI agent and application developers a proper home in CDF, and wanted to share the approach we're exploring in case it's useful to others facing the same thing.The problem we hitA standard dev / test / prod setup works well for governed data pipelines, but it gets awkward for AI agent and app development, which needs two things that are hard to provide together in those environments:Representative production data — dev is typically fed by a non-representative subset, so agents and apps built there don't behave the same once they meet real data. Broad, globally-scoped rights to create and edit agents and apps. Since creation rights are global within a project and can't be scoped down to a space — even with Row-Level Security — granting them in a shared dev project exposes every other workload there.The approach we're exploringA dedicated CDF environment (separate project) running parallel t

Hi community!Cognite has just announced Flows Custom Applications (Dune). If you’ve been using the Dune documentation, please note that it’s now available at the updated link below:https://docs.cognite.com/cdf/flows 

Hello all,We are trying to establish connectivity to a source which has cassadra as its backend database. The system allows OPCUA connection but we will have only live data feed that way, without access to historical data.Has anyone explored or tested data extarction from Cassandra. Any inputs would be valuable. Thanks 

I posted it in Github, but cross-posting here too.https://github.com/cognitedata/cognite-sdk-python/issues/2525 System information (please complete the following information):OS: Windows11 Python Version: 3.12 SDK Version: 8.Describe the bugThis code previously worked:NodeId.load_if(None)But since v8, I am getting this error:AttributeError: type object 'NodeId' has no attribute 'load_if'. Did you mean: '_load_if'?To ReproduceRunnable code reproducing the error.NodeId.load_if(None)Expected behaviorIn Cognite SDK v7 this returns None.  

How much time does it really take to build a full solution?We deployed a complete solution using Cognite CDF, Cursor, and Anthropic Claude Opus 4.6, powered heavily by Gen AI in just 4 hours.This was not just a prototype screen. It was a fully deployed solution for demo purpose.And here’s the most interesting part:All of the following steps were done using Gen AI:1️⃣ Creation of the data model in Cognite Data Fusion and the Terraform2️⃣ Deployment of the model in Cognite3️⃣ Data generation and ingestion into the Cognite Data Platform4️⃣ Development of a React application on top of Cognite using the Cognite SDK5️⃣ Creation of: Asset Map, KPIs, Production Time Series Dashboardand Well Intervention viewFrom data modeling to frontend application — accelerated with AI.This is not about replacing engineers.It’s about dramatically increasing speed, experimentation, and delivery capacity. The question is no longer “Can we build it?”It’s “How fast can we build it?”Post here!  

Hello, I was playing with this project (https://github.com/blurrah/mcp-graphql) and created an MCP server on top of a test CDF data model. You can then use an LLM that supports MCP clients to register your server and have the help of the LLM to analyse your data :) Here I used Claude desktop for example:  ​​​​​​This is a great way if we want to integrate CDF data models with in-house or local LLM models.Do you plan to release an official MCP server for CDF data models? Thank you,

Hi everyone,I’d like to share industrial-model — a Python ORM built on top of the Cognite Data Modeling Service API.It lets you define DMS views using Pydantic models, query them with a fluent, expressive API (filter, search, aggregate), and get fully typed results with IDE autocomplete out of the box.The SDK also supports upserts, deletes, and async workflows, making it a natural fit for modern Python stacks. In addition, it includes configuration for injecting instance spaces directly into queries, which can significantly improve query performance when working with large or complex data models.We’ve been using industrial-model for the past couple of months, and it has significantly reduced boilerplate when querying complex graphs, while also speeding up onboarding for new developers.Documentation:https://github.com/lucasrosaalves/industrial-model Feedback and feature requests are very welcome!Sample code:from pathlib import Pathfrom industrial_model import ( Engine, ViewInstanc

The documentation on this page should be updated to reflect the product behavior.https://api-docs.cognite.com/20230101/tag/Spaces/operation/deleteSpacesV3> If an existing data model references a space, you cannot delete that space. Nodes, edges and other data types that are part of a space will no longer be available. I tried deleting a space with a single node and I get this error:cognite.client.exceptions.CogniteAPIError: Unable to delete spaces because they contain nodes or edges: [some-test-space] | code: 400 This does not exactly contradict the documentation, but it means that the documentation is misleading. As it is written, it suggests a space can be used to manage lifecycle of ephemeral test data.

Wow—was I in for a challenge!I just received my first proper assignment after joining Cognite: building a custom extractor to move documents and their metadata from a Document Management System (DMS) into Cognite Data Fusion (CDF).Diving headfirst into this, I found myself at the intersection of a steep learning curve, the high-stakes data needs of our industrial customers, and my own belief in Agile engineering practices. Coming from a background where Test-Driven Development (TDD) is the heartbeat of quality, I realized I didn't just want to build a script; I wanted to build a process I could trust.In this series, I want to share a practice that served me well while learning the ropes: The "Twin Auditor" Pattern.The Challenge: Beyond Logic, It’s IntegrationWhen you build an extractor, you aren't just writing code in a vacuum. You are building a bridge between two distinct worlds: your source system (the DMS) and your data platform (CDF).Extraction is inherently an integration challen

 We are thrilled to announce the next evolution of your user experience. We’ve unified our ecosystem to help you master your industrial data without the friction.What’s new? One Identity: We’ve merged Cognite Academy and the Community. One login for everything. Unified Search: Find How-to guides, discussions, and courses in a single search. Guided Careers: New role-based learning paths, starting with the Data Engineer path! 👇 Read the full announcement here:  

Hi everyone,As we prepare for the General Availability (GA) release of Records in CDF, we're implementing important changes based on learnings from the Private Beta. These changes will take effect on November 3rd, 2025.What's ChangingStream Limits per CDF Project:Active streams: reduced from 10 to 3 Soft-deleted streams: reduced from 100 to 30Stream Templates: We're streamlining from 6 private beta-phase templates to 3 templates:ImmutableTestStream - for experimentation only BasicArchive - for perpetual data storage (immutable) BasicLiveData - for production usage (mutable)The following beta templates will no longer be available for new stream creation:ImmutableDataStaging ImmutableNormalizedData ImmutableArchive MutableTestStream MutableLiveDataIf you're using ImmutableTestStream, note that the template capacity has been significantly reduced. The maximum total records decreased from 5B to 50M, and the maximum total data volume decreased from 500GB to 50GB.Existing streams will retain

Hi,I have been updating the beta documentation a bit since the last edition linked in the invitation. This applies to both the planned standard Cognite documentation, and the developer API documentation. All of the documents are available using a direct link to our document rendering services (linked below), and should be updated as we privately deploy new information.Note that all of these documents are works in progress with ongoing updates, so please forgive any typos, omissions, and other errors at this stage: Streams API documentation Records API documentation Capabilities for CDF Records Updated Data Modeling concepts page Concepts page for CDF Records (and Streams) Example high level use case (alarms) for CDF RecordsPlease do not share these documents.

As illustrated in our documentation, the CDF Records feature uses a Data Modeling container as the schema definition for record data. I.e. you have to create a space and container first, before you start loading Records to a Stream. (Note: It is possible to use multiple containers together to define the schema for a single record. This may make sense in the context of, for instance, a Work Order record).However, at the moment when I write this, using the container based schema represents a somewhat confusing “limitation” when it comes to the size of a container vs the size of a record (number of properties). The way we have implemented this capability in CDF Records at the moment - using the DM containers - comes with a side-effect: The number of properties you can have for a single container is, as of right now, the same as it is in your Data Modeling service.The limits we're documenting in terms of properties for CDF Records are linked to the properties containing data within a singl

All,Now that you’ve had some time to “play with” the Records feature, we’re looking to identify what the consensus is when it comes to the available aggregations for Records. Do we have a useful mix of aggregations available in the API today? Which of the aggregates do you use most often? What is your experience with the API documentation for aggregate API endpoint so far? Have you attempted to use the movingFunction aggregate? What problem did you use it to solve for your  use case? Was the information in the documentation sufficient and useful for you? If you haven’t used the aggregate, please help us understand why not? Have you attempted to use the timeHistogram aggregate? Was the information in the documentation sufficient and useful for you? If you haven’t used the aggregate, please help us understand why not? Is there a use case involving aggregates that you have not been able to address in Records? Please describe the use case? How important is the “missing” aggregate for

Had a chance to experiment with streams and records and it seems to be working well.  Couple of questions based on what I found:Are any of the filtering options that are available for general data modeling queries but not available with records likely to be supported in future?  Specifically, I am thinking of queries like Fetch me all of the records in the last week where the `asset` property is below `Facility-ABC` (i.e. it contains `Facility-ABC` in asset.path).  I think this would require us to use the nested filter unless the total number of assets that were below Facility-ABC was small enough to pass them directly into the filter.  I can imagine that this kind of filter is more difficult to implement and a more expensive operation, but I think it could be valuable. Is there a reason why containers rather than views must be used when creating/querying records?  It seems like a view with a specific version is nothing more than a set of containers with a (possibly incomplete) list of

Hi,We have just rolled out “mutability” support for the Records API service. Mutability is the ability to change a record once it has initially been written to the Records API service.Enabling mutability for a stream requires using the settings.template.name key in the payload of the creation request for a stream. I.e. submitting a POST operation to the /streams endpoint, with - for example - the following body:items: [ externalId: "a-mutable-stream-1", settings: { template: { name: "MutableTestStream" } }] There are two supported “mutable”stream settings templates: “MutableTestStream”, and “MutableLiveData”.To update or create a record in this stream, you must use the newly introduced upsert endpoint in the Records API and specify the required identifiers of the previously ingested record you’re wanting to update.Over the next couple of weeks, we would love it if you could spend some time familiarizing yourself with mutable streams, and test record updates to he

We intend to effect 3 breaking changes to the Records API over the next couple of weeks of the Private Beta program. These changes may require updates of your test procedures. /streams API “settings” attribute will be required Summary: Modifying the Cognite Streams API.From when: After July 15, 2025Description:The Streams API provides a broad spectrum of functionalities, but it's important to understand that these capabilities are not mutually exclusive; enhancing one often means adjusting another. For instance, if a stream is designed for permanent data storage, it will offer unlimited record retention but a lower maximum ingestion rate. Conversely, streams built for temporary data staging will support significantly bigger ingestion rate but only for a brief retention period. Similarly, you'll choose between mutable streams (allowing record changes) and immutable streams (optimized for high volume and speed).Because of these crucial distinctions, it's essential for users to be fully a

ContextQuerying views with a large number of instances (>1 million), we frequently encounter query timeout issues. This has become a critical bottleneck affecting application performance and user experience. To mitigate this, we introduced on the app layer a pre-query caching strategy:Before sending a query to Cognite, we aggregate the number of instance spaces for a given view using the endpoint /models/instances/aggregate. This result is stored in a cache layer. When a query is initiated, we check if the user included a space filter. If not, we append the known relevant spaces from the cache to the query filter. This approach has significantly reduced timeouts across our applications. However, it introduces new challenges:One request per view is still needed to fetch associated spaces. Cache invalidation must be managed periodically, especially as user capabilities may change. This workaround does not help with timeouts in the CDF UI or Infield tools, where we cannot control the q

HiI am looking into using the toolkit more actively for deploying resources to CDF. One question that was raised when researching how to use the toolkit is what kind of validations actually happens when doing a dry run for deploying data modeling resources. I do not really have any specific issue I want answered, but rather want to learn more about the tool so prepare for a lot of questions from my notes:)Does it test that the configuration of views and containers work together?Does a successful dry run mean that I can be sure that the deployment will always work?Are there anything I need to consider even after getting a successful dry run?What kind of responses do I get if the dry run finds that something is wrong? Do I get any hints about how to fix an issue?Does it consider what is already deployed into the CDF environment?Will it tell me about any issues that can happen with new breaking changes? Appreciate all kinds of insights and experiences around this topic :)Sebastian

I am able to Delete data using python code using Primary key of the table in CDF staging/RAW. But I need help with deleting data based on where condition for columns other than primary key. I am following the below documentation for deleting based on primary key. Data Ingestion — cognite-sdk 7.74.5 documentation  Delete rows from table:>>> from cognite.client import CogniteClient>>> client = CogniteClient()>>> keys_to_delete = ["k1", "k2", "k3"]>>> client.raw.rows.delete("db1", "table1", keys_to_delete)

can someone please explain the this procedure for me with details ? because I got really confused. so first we register an app  and create a client secret and add api read all permission n Microsoft azure, after that we create an app in SharePoint  and we add the “permission request” to it later. after that what is next before downloading the extractor ? how are the two apps linked ? I saw someone at my work using Microsoft graph to link them but did not understand the logic behind it. and if this how they are linked , how ?  when in the Microsoft when doing the post each app dose not mention the other  

I am starting the Cognite Data Engineer Learning Path. How to get the access to Cognite Data Fusion platform to practice the learning.

Hi Everyone,I need to create query like it is creating on cognite UI’s  like   { "listEntity": { "with": { "0": { "limit": 50, "nodes": { "filter": { "and": [ { "matchAll": {} }, { "hasData": [ { "type": "view", "space": "slb-pdm-dm-governed", "externalId": "Entity", "version": "1_7" } ] }, { "or": [ { "and": [ { "nested": { "scope": [ "slb-pdm-dm-governed", "Entity/1_7", "parent" ],

I am trying to update column value in view using upsert method, but it's not working and giving 400 error code.  adding screenshot of upsert api, as its not matching with the input structure provided in document.