Community Discussions

Hi Cognite Community,We will be performing scheduled upgrade today from 4:00 PM to 8:00 PM CEST. Our platform will be temporarily unavailable. We appreciate your patience.

As illustrated in our documentation, the CDF Records feature uses a Data Modeling container as the schema definition for record data. I.e. you have to create a space and container first, before you start loading Records to a Stream. (Note: It is possible to use multiple containers together to define the schema for a single record. This may make sense in the context of, for instance, a Work Order record).However, at the moment when I write this, using the container based schema represents a somewhat confusing “limitation” when it comes to the size of a container vs the size of a record (number of properties). The way we have implemented this capability in CDF Records at the moment - using the DM containers - comes with a side-effect: The number of properties you can have for a single container is, as of right now, the same as it is in your Data Modeling service.The limits we're documenting in terms of properties for CDF Records are linked to the properties containing data within a singl

H ​@Mithila Jayalath ,Can you please grant relationship API permissions in the Python SDK?I am getting this error:Could not fetch relationships: User does not have relationships permissions. code: 403 | X-Request-ID: 10dc4192-f4f6-91e1-9c0c-929eefa8bd15 | cluster: api | project: publicdatacdm

Hi Everyone,I'm part of the performance team, and I want to learn more about transformation.My main goals are:To understand the basic way it works. To see how each step interacts with different parts, so I can troubleshoot problems better. To find out what kinds of scenarios are commonly used with customers. To know which metrics we are currently monitoring in Grafana to check for issues like server errors (5xx), high CPU usage, or memory problems. To learn where I can see live logs. To understand which customer is the biggest right now and what’s the largest amount of data used for transformation so far.

All,Now that you’ve had some time to “play with” the Records feature, we’re looking to identify what the consensus is when it comes to the available aggregations for Records. Do we have a useful mix of aggregations available in the API today? Which of the aggregates do you use most often? What is your experience with the API documentation for aggregate API endpoint so far? Have you attempted to use the movingFunction aggregate? What problem did you use it to solve for your  use case? Was the information in the documentation sufficient and useful for you? If you haven’t used the aggregate, please help us understand why not? Have you attempted to use the timeHistogram aggregate? Was the information in the documentation sufficient and useful for you? If you haven’t used the aggregate, please help us understand why not? Is there a use case involving aggregates that you have not been able to address in Records? Please describe the use case? How important is the “missing” aggregate for

Had a chance to experiment with streams and records and it seems to be working well.  Couple of questions based on what I found:Are any of the filtering options that are available for general data modeling queries but not available with records likely to be supported in future?  Specifically, I am thinking of queries like Fetch me all of the records in the last week where the `asset` property is below `Facility-ABC` (i.e. it contains `Facility-ABC` in asset.path).  I think this would require us to use the nested filter unless the total number of assets that were below Facility-ABC was small enough to pass them directly into the filter.  I can imagine that this kind of filter is more difficult to implement and a more expensive operation, but I think it could be valuable. Is there a reason why containers rather than views must be used when creating/querying records?  It seems like a view with a specific version is nothing more than a set of containers with a (possibly incomplete) list of

 I have created 1 workflow , in which I am creating dynamic tasks depending on input, it creates batch of ids and create tasks out of it. Below is workflow definitionWorkflowVersionUpsert(   workflow_external_id="test_dynamic-0729",   version="1",   workflow_definition=WorkflowDefinitionUpsert(       description="This workflow has two steps",       tasks=[           WorkflowTask(                external_id="test_sub_tasks",                parameters=FunctionTaskParameters(                    external_id="test_sub_tasks",                    data="${workflow.input}"                ),                retries=1,                timeout=3600,                depends_on=[],                on_failure = "abortWorkflow",            ),           WorkflowTask(                external_id="test_create_sub",                parameters=DynamicTaskParameters(                    tasks="${test_sub_tasks.output.response.tasks}"                ),                name="Dynamic Task",                description=

Hi,We have just rolled out “mutability” support for the Records API service. Mutability is the ability to change a record once it has initially been written to the Records API service.Enabling mutability for a stream requires using the settings.template.name key in the payload of the creation request for a stream. I.e. submitting a POST operation to the /streams endpoint, with - for example - the following body:items: [ externalId: "a-mutable-stream-1", settings: { template: { name: "MutableTestStream" } }] There are two supported “mutable”stream settings templates: “MutableTestStream”, and “MutableLiveData”.To update or create a record in this stream, you must use the newly introduced upsert endpoint in the Records API and specify the required identifiers of the previously ingested record you’re wanting to update.Over the next couple of weeks, we would love it if you could spend some time familiarizing yourself with mutable streams, and test record updates to he

We intend to effect 3 breaking changes to the Records API over the next couple of weeks of the Private Beta program. These changes may require updates of your test procedures. /streams API “settings” attribute will be required Summary: Modifying the Cognite Streams API.From when: After July 15, 2025Description:The Streams API provides a broad spectrum of functionalities, but it's important to understand that these capabilities are not mutually exclusive; enhancing one often means adjusting another. For instance, if a stream is designed for permanent data storage, it will offer unlimited record retention but a lower maximum ingestion rate. Conversely, streams built for temporary data staging will support significantly bigger ingestion rate but only for a brief retention period. Similarly, you'll choose between mutable streams (allowing record changes) and immutable streams (optimized for high volume and speed).Because of these crucial distinctions, it's essential for users to be fully a

SUMMARY Committed to elevating our service and support standards, we continually evaluate and refine our processes to improve the exchange of information between end-users and the Cognite support team. We have identified that including additional information upfront in the support tickets raised can greatly streamline our response process, allowing for quicker and more efficient issue resolution. To facilitate this, we have pinpointed essential details that, when provided in tickets, can significantly accelerate resolution times.We are dedicated to continuous improvement in these areas and greatly value your feedback.  There is basic information that will be helpful for troubleshooting Have you encountered this issue before, or is this the first time you have experienced it? To better understand the impact, please let us know the number of affected users. What is the project name? The following questions will be added based on the product: SUMMARY Charts Questions  Functions Questions

Hi,I would like to update data to the EDR. The EDR has a build in tool to upload data, but I would like to use the API. As I understand it is not officially supported, but we would like to give it a try as we have hunderds of files we would like to upload.Before I asked my questions by e-mail, but I am directed to this hub to ask my questions, so there we go: File upload:I'm using the client.files.upload() function [1]. It seems to work when only the file path is provided, but I'm unsure whether this is sufficient or if I should be populating more of the optional arguments. Here's what I'm looking at: path external_id name source mime_type metadata directory asset_ids source_created_time source_modified_time data_set_id labels geo_location security_categories recursive overwrite  name: It appears this is automatically assigned based on the filename, is that okay?directory: This seems to default to the dataset ID, but I’ve run into an error when using 968ca

Whose idea was it to hide the dropdown behind the filters? 

Good news, innovators!We've heard your requests and have extended the deadline for registration, team formation, and idea submission for the Impact Challenge 2025. You now have until this Friday, June 13th, at the end of the day (US time) to get your ideas in.Don't miss this opportunity to solve real-world challenges with Cognite Data Fusion®, win amazing prizes, and present your solution at the Impact Conference in Houston.No coding required!👉 Join the Challenge Now: https://hub.cognite.com/p/hack-for-impactLet's turn your ideas into impact!

Dear Cognite Hub community: We are very happy to announce that our Cognite SAP Extractor is now generally available to CDF users!The extractor connects to OData V2.0 endpoints in the SAP NetWeaver Gateway, making use of many pre-built integrations available in S/4HANA and saving considerable implementation time, while also natively connecting to the SAP data you need in a standardised manner.To get started,  download the SAP Extractor from the “Extract Data” page directly from CDF. The documentation, including Server Requirements and how to set up your SAP extractor is available here.We have also prepared a short demo (less than 4 minutes long) showing the steps needed to setup a data extraction from SAP S/4HANA OnPremise to CDF from scratch, including how to find the standard service from SAP, enable it in S/4HANA, configure and run the extractor and then see the SAP data in CDF.   Any feedback is very welcome, after testing the extractor and/or watching the quick demo please make sur

ContextQuerying views with a large number of instances (>1 million), we frequently encounter query timeout issues. This has become a critical bottleneck affecting application performance and user experience. To mitigate this, we introduced on the app layer a pre-query caching strategy:Before sending a query to Cognite, we aggregate the number of instance spaces for a given view using the endpoint /models/instances/aggregate. This result is stored in a cache layer. When a query is initiated, we check if the user included a space filter. If not, we append the known relevant spaces from the cache to the query filter. This approach has significantly reduced timeouts across our applications. However, it introduces new challenges:One request per view is still needed to fetch associated spaces. Cache invalidation must be managed periodically, especially as user capabilities may change. This workaround does not help with timeouts in the CDF UI or Infield tools, where we cannot control the q

HiI am looking into using the toolkit more actively for deploying resources to CDF. One question that was raised when researching how to use the toolkit is what kind of validations actually happens when doing a dry run for deploying data modeling resources. I do not really have any specific issue I want answered, but rather want to learn more about the tool so prepare for a lot of questions from my notes:)Does it test that the configuration of views and containers work together?Does a successful dry run mean that I can be sure that the deployment will always work?Are there anything I need to consider even after getting a successful dry run?What kind of responses do I get if the dry run finds that something is wrong? Do I get any hints about how to fix an issue?Does it consider what is already deployed into the CDF environment?Will it tell me about any issues that can happen with new breaking changes? Appreciate all kinds of insights and experiences around this topic :)Sebastian

Aker Solutions Verdal Production Line (VPL)Last week I had the chance to visit the Aker Solutions team at their impressive yard in Verdal, fabricating fit-for-purpose steel substructures and jackets for offshore developments, and a great example of industrial innovation done right.At the heart of the yard is the Verdal Production Line (VPL), a fully robotized line that opened in 2024. With automated welding, sandblasting, and painting, VPL is delivering production speeds up to 10x faster than traditional methods, cutting costs and improving safety by keeping people away from hazardous tasks.To push things even further, the team has developed their own software for weld planning. The robots scan each pipe, create a 3D model, and then autonomously plan and carry out the welding.Excited to follow VPL further. Aker Robotics 🦾Aker Solutions - Yards and Fabrication📸 Johan Arnt Nesgård 

I am able to Delete data using python code using Primary key of the table in CDF staging/RAW. But I need help with deleting data based on where condition for columns other than primary key. I am following the below documentation for deleting based on primary key. Data Ingestion — cognite-sdk 7.74.5 documentation  Delete rows from table:>>> from cognite.client import CogniteClient>>> client = CogniteClient()>>> keys_to_delete = ["k1", "k2", "k3"]>>> client.raw.rows.delete("db1", "table1", keys_to_delete)

Hello Team,I have a modeling question.We want to add some well information to all object types in our knowledge graph. At the same time, we don't want to create a link from the objects to the well.So, to avoid data duplication and updating these pieces of information, I thought about using the mapping functionality.For example, I have a casing object with two attributes: name and diameter. And I added a name_well attribute using the mapping functionality, which, as I understand, under the hood, performs a join.type Well { name: String}type Casing { name: String name_well: String @mapping(container: "Well", property: "name") diameter: Float32}Well transformation:select wh.IDWELL as externalId, wh.WELLNAME as namefrom well-data as whCasing transformationselect cas.IDCASING AS externalId, cas.NAME AS name, cas.DIAMETER as diameter, wh.WELLNAME as name_wellfrom casing-data as cas join well-data as wh on cas.IDWELL == cas.IDWELLWhat I don’t understand is how exactly this join i

can someone please explain the this procedure for me with details ? because I got really confused. so first we register an app  and create a client secret and add api read all permission n Microsoft azure, after that we create an app in SharePoint  and we add the “permission request” to it later. after that what is next before downloading the extractor ? how are the two apps linked ? I saw someone at my work using Microsoft graph to link them but did not understand the logic behind it. and if this how they are linked , how ?  when in the Microsoft when doing the post each app dose not mention the other  

I am starting the Cognite Data Engineer Learning Path. How to get the access to Cognite Data Fusion platform to practice the learning.

The new CDF Power BI REST connector has been certified by Microsoft and is now included in the latest Power BI desktop version and deployed in the Power BI online service.What's new with the Power BI REST connector:Flexible authentication: Connect Power BI with any IdP supported by CDF (the legacy OData connector only worked with Azure Entra ID) Broader data access: Fetch data from OData services (just like the legacy connector) Access data from Data Models using GraphQL Connect to any GA CDF API endpoint Significant performance boost: Up to 10x faster when using regular REST endpoints compared to fetching the same data via ODataThe connector is currently in Beta, and we're eager to hear customer feedback before promoting it to GA. The documentation for the new connector is available here, and we're working on a new set of micro learning modules in Academy based on the new connector.

In Open Industrial Data (OID), we have moved away from API keys. Open Industrial Data currently supports Open ID connect. You can check out more on how to configure OpenID Connect on Open Industrial Data.If you are planning to use OID and authenticate using client credentials flow, you will need a client secret from the app registration on the Azure Active Directory. Go to Open Industrial Data and you will observe there is a widget for generating a new client secret IDIn the drop down, you can select two options:Other: Use this if you are using Postman or Python SDK Javascript: Select this option if your app is in JavascriptOnce you click on Create client secret, this will be display just once. Make sure to save it somewhere safe.Let me know if you have any questions 🙂

Is anyone using Cognite as their main timeseries historian? We are always exploring alternatives and would be interested to hear if Cognite has fit this use case for any users. 

Here are the features and functionalities that you can expect to find in Industrial Canvas as of June release:Work with several related data types:You can search and add the data you need in the canvas such as files, images, time series, assets, events. Use the button “Add data”. When adding P&IDs to the canvas, each highlighted area is clickable and can be used to add additional data to the canvas. Add personal files and images from your local machine by just using drag-and-drop. You can also add data to the canvas while being in Data Explorer, in Cognite Data Fusion by clicking on “Open in Industrial Canvas”.Add related data directly from the data already selected in the canvas  User interactivity:Create/Rename/Delete a canvas. Free form experience, canvas-like experience. You have access to different shapes, text box, sticky notes and connected lines. Easily add insights on top of selected data.Collaborate with your peers:Share the canvas with your colleagues by either sharing t