Community Discussions

 Issue SummaryCreating `CogniteFile` instances in Data Modeling Service (DMS) automatically creates unwanted "ghost files" - duplicate file resources that are incomplete and have no external_id, no dataset_id, and no content. This causes the Diagram Parser UI to display incorrect file IDs and show "Files not uploaded" errors.  Environment- Project: tridcognite-sanbox- Cluster: aw-was-gp-001.cognitedata.com- Dataset ID: 1242342447467142- Space: diagram_parser_space_1- Data Model: diagram_parser_data_model_1 (version 1)- Python SDK Version: 7.78.0---Methods Used:Method 1: SQL TransformationINSERT INTO cdf_cdm.CogniteFile(v1)SELECT   CAST(externalId AS STRING) as externalId,  CAST(name AS STRING) as name,  CAST('application/pdf' AS STRING) as mimeType,  CAST(TRUE AS BOOLEAN) as isUploadedFROM `Diag_Parser_Cement`.`parser_files_staging`WHERE externalId = 'cement-parser-file-crusher-pid' Transformation Configuration:- Destination: Instances- Space: diagram_parser_space_1- View: cdf_cdm.Cogn

Hello,When testing the new Diagram Parser tool on Vector PDFs, I found that it successfully finds some Assets ingested in the DMS and Symbols from a selected library. However, the Assets only show up as annotations even when verifying them. How can I get the suggested Assets to link to the Diagram in the DMS (i.e. open the diagram and see Assets directly linked to the diagram)? Thanks,Gabriel

Why are we making this change?The CDF CLI was originally designed to streamline application development on top of CDF, specifically focusing on Data Modeling workflows and the generation of type-safe SDKs.However, to ensure we are delivering the highest value to our developer ecosystem, we are refining our product strategy and priorities. The CDF CLI (beta) has not been under active development for some time, and maintaining it to our quality standards no longer aligns with our current roadmap. By discontinuing this tool, we can refocus our resources on the core tools and SDKs that best support your integration with Cognite Data Fusion.If your current CI/CD pipelines or local development environments rely on installing this package, you must take action before February 18th, 2026 to avoid build failures.Timeline of Events February 18th, 2026: The @cognite/cdf-cli package will be unpublished from NPM. You will no longer be able to install it via npm install or yarn add. What do you need

Hi everyone,As we prepare for the General Availability (GA) release of Records in CDF, we're implementing important changes based on learnings from the Private Beta. These changes will take effect on November 3rd, 2025.What's ChangingStream Limits per CDF Project:Active streams: reduced from 10 to 3 Soft-deleted streams: reduced from 100 to 30Stream Templates: We're streamlining from 6 private beta-phase templates to 3 templates:ImmutableTestStream - for experimentation only BasicArchive - for perpetual data storage (immutable) BasicLiveData - for production usage (mutable)The following beta templates will no longer be available for new stream creation:ImmutableDataStaging ImmutableNormalizedData ImmutableArchive MutableTestStream MutableLiveDataIf you're using ImmutableTestStream, note that the template capacity has been significantly reduced. The maximum total records decreased from 5B to 50M, and the maximum total data volume decreased from 500GB to 50GB.Existing streams will retain

As of January 1st, 2026, the Rule-Based Coloring feature in Industrial Canvas will transition from public beta to private beta. The change will not affect existing users who have previously enabled and are using the feature on a daily basis. The feature will be unavailable for new and existing users who are not in the private beta program.No action is needed. If you have any questions or concerns, please reach out.Thank you,Andreea. 

The “Ask a Question” feature inside the Canvas is very helpful—it allows users to quickly access specific documents and find information easily.However, I noticed that this feature is not available in the Industrial Tool – Search option or under the Data Management – Data Explorer tab. It would be very useful to have this functionality directly within the Data Explorer, as it would make the process more efficient and user-friendly without needing to switch to Canvas.What are your thoughts on this?Is it possible to consider adding this feature to Data Explorer in the future?

Hi,I have been updating the beta documentation a bit since the last edition linked in the invitation. This applies to both the planned standard Cognite documentation, and the developer API documentation. All of the documents are available using a direct link to our document rendering services (linked below), and should be updated as we privately deploy new information.Note that all of these documents are works in progress with ongoing updates, so please forgive any typos, omissions, and other errors at this stage: Streams API documentation Records API documentation Capabilities for CDF Records Updated Data Modeling concepts page Concepts page for CDF Records (and Streams) Example high level use case (alarms) for CDF RecordsPlease do not share these documents.

Hi Cognite Community,We will be performing scheduled upgrade today from 4:00 PM to 8:00 PM CEST. Our platform will be temporarily unavailable. We appreciate your patience.

As illustrated in our documentation, the CDF Records feature uses a Data Modeling container as the schema definition for record data. I.e. you have to create a space and container first, before you start loading Records to a Stream. (Note: It is possible to use multiple containers together to define the schema for a single record. This may make sense in the context of, for instance, a Work Order record).However, at the moment when I write this, using the container based schema represents a somewhat confusing “limitation” when it comes to the size of a container vs the size of a record (number of properties). The way we have implemented this capability in CDF Records at the moment - using the DM containers - comes with a side-effect: The number of properties you can have for a single container is, as of right now, the same as it is in your Data Modeling service.The limits we're documenting in terms of properties for CDF Records are linked to the properties containing data within a singl

H ​@Mithila Jayalath ,Can you please grant relationship API permissions in the Python SDK?I am getting this error:Could not fetch relationships: User does not have relationships permissions. code: 403 | X-Request-ID: 10dc4192-f4f6-91e1-9c0c-929eefa8bd15 | cluster: api | project: publicdatacdm

Hi Everyone,I'm part of the performance team, and I want to learn more about transformation.My main goals are:To understand the basic way it works. To see how each step interacts with different parts, so I can troubleshoot problems better. To find out what kinds of scenarios are commonly used with customers. To know which metrics we are currently monitoring in Grafana to check for issues like server errors (5xx), high CPU usage, or memory problems. To learn where I can see live logs. To understand which customer is the biggest right now and what’s the largest amount of data used for transformation so far.

All,Now that you’ve had some time to “play with” the Records feature, we’re looking to identify what the consensus is when it comes to the available aggregations for Records. Do we have a useful mix of aggregations available in the API today? Which of the aggregates do you use most often? What is your experience with the API documentation for aggregate API endpoint so far? Have you attempted to use the movingFunction aggregate? What problem did you use it to solve for your  use case? Was the information in the documentation sufficient and useful for you? If you haven’t used the aggregate, please help us understand why not? Have you attempted to use the timeHistogram aggregate? Was the information in the documentation sufficient and useful for you? If you haven’t used the aggregate, please help us understand why not? Is there a use case involving aggregates that you have not been able to address in Records? Please describe the use case? How important is the “missing” aggregate for

Had a chance to experiment with streams and records and it seems to be working well.  Couple of questions based on what I found:Are any of the filtering options that are available for general data modeling queries but not available with records likely to be supported in future?  Specifically, I am thinking of queries like Fetch me all of the records in the last week where the `asset` property is below `Facility-ABC` (i.e. it contains `Facility-ABC` in asset.path).  I think this would require us to use the nested filter unless the total number of assets that were below Facility-ABC was small enough to pass them directly into the filter.  I can imagine that this kind of filter is more difficult to implement and a more expensive operation, but I think it could be valuable. Is there a reason why containers rather than views must be used when creating/querying records?  It seems like a view with a specific version is nothing more than a set of containers with a (possibly incomplete) list of

 I have created 1 workflow , in which I am creating dynamic tasks depending on input, it creates batch of ids and create tasks out of it. Below is workflow definitionWorkflowVersionUpsert(   workflow_external_id="test_dynamic-0729",   version="1",   workflow_definition=WorkflowDefinitionUpsert(       description="This workflow has two steps",       tasks=[           WorkflowTask(                external_id="test_sub_tasks",                parameters=FunctionTaskParameters(                    external_id="test_sub_tasks",                    data="${workflow.input}"                ),                retries=1,                timeout=3600,                depends_on=[],                on_failure = "abortWorkflow",            ),           WorkflowTask(                external_id="test_create_sub",                parameters=DynamicTaskParameters(                    tasks="${test_sub_tasks.output.response.tasks}"                ),                name="Dynamic Task",                description=

Hi,We have just rolled out “mutability” support for the Records API service. Mutability is the ability to change a record once it has initially been written to the Records API service.Enabling mutability for a stream requires using the settings.template.name key in the payload of the creation request for a stream. I.e. submitting a POST operation to the /streams endpoint, with - for example - the following body:items: [ externalId: "a-mutable-stream-1", settings: { template: { name: "MutableTestStream" } }] There are two supported “mutable”stream settings templates: “MutableTestStream”, and “MutableLiveData”.To update or create a record in this stream, you must use the newly introduced upsert endpoint in the Records API and specify the required identifiers of the previously ingested record you’re wanting to update.Over the next couple of weeks, we would love it if you could spend some time familiarizing yourself with mutable streams, and test record updates to he

We intend to effect 3 breaking changes to the Records API over the next couple of weeks of the Private Beta program. These changes may require updates of your test procedures. /streams API “settings” attribute will be required Summary: Modifying the Cognite Streams API.From when: After July 15, 2025Description:The Streams API provides a broad spectrum of functionalities, but it's important to understand that these capabilities are not mutually exclusive; enhancing one often means adjusting another. For instance, if a stream is designed for permanent data storage, it will offer unlimited record retention but a lower maximum ingestion rate. Conversely, streams built for temporary data staging will support significantly bigger ingestion rate but only for a brief retention period. Similarly, you'll choose between mutable streams (allowing record changes) and immutable streams (optimized for high volume and speed).Because of these crucial distinctions, it's essential for users to be fully a

SUMMARY Committed to elevating our service and support standards, we continually evaluate and refine our processes to improve the exchange of information between end-users and the Cognite support team. We have identified that including additional information upfront in the support tickets raised can greatly streamline our response process, allowing for quicker and more efficient issue resolution. To facilitate this, we have pinpointed essential details that, when provided in tickets, can significantly accelerate resolution times.We are dedicated to continuous improvement in these areas and greatly value your feedback.  There is basic information that will be helpful for troubleshooting Have you encountered this issue before, or is this the first time you have experienced it? To better understand the impact, please let us know the number of affected users. What is the project name? The following questions will be added based on the product: SUMMARY Charts Questions  Functions Questions

Hi,I would like to update data to the EDR. The EDR has a build in tool to upload data, but I would like to use the API. As I understand it is not officially supported, but we would like to give it a try as we have hunderds of files we would like to upload.Before I asked my questions by e-mail, but I am directed to this hub to ask my questions, so there we go: File upload:I'm using the client.files.upload() function [1]. It seems to work when only the file path is provided, but I'm unsure whether this is sufficient or if I should be populating more of the optional arguments. Here's what I'm looking at: path external_id name source mime_type metadata directory asset_ids source_created_time source_modified_time data_set_id labels geo_location security_categories recursive overwrite  name: It appears this is automatically assigned based on the filename, is that okay?directory: This seems to default to the dataset ID, but I’ve run into an error when using 968ca

Whose idea was it to hide the dropdown behind the filters? 

Good news, innovators!We've heard your requests and have extended the deadline for registration, team formation, and idea submission for the Impact Challenge 2025. You now have until this Friday, June 13th, at the end of the day (US time) to get your ideas in.Don't miss this opportunity to solve real-world challenges with Cognite Data Fusion®, win amazing prizes, and present your solution at the Impact Conference in Houston.No coding required!👉 Join the Challenge Now: https://hub.cognite.com/p/hack-for-impactLet's turn your ideas into impact!

Dear Cognite Hub community: We are very happy to announce that our Cognite SAP Extractor is now generally available to CDF users!The extractor connects to OData V2.0 endpoints in the SAP NetWeaver Gateway, making use of many pre-built integrations available in S/4HANA and saving considerable implementation time, while also natively connecting to the SAP data you need in a standardised manner.To get started,  download the SAP Extractor from the “Extract Data” page directly from CDF. The documentation, including Server Requirements and how to set up your SAP extractor is available here.We have also prepared a short demo (less than 4 minutes long) showing the steps needed to setup a data extraction from SAP S/4HANA OnPremise to CDF from scratch, including how to find the standard service from SAP, enable it in S/4HANA, configure and run the extractor and then see the SAP data in CDF.   Any feedback is very welcome, after testing the extractor and/or watching the quick demo please make sur

ContextQuerying views with a large number of instances (>1 million), we frequently encounter query timeout issues. This has become a critical bottleneck affecting application performance and user experience. To mitigate this, we introduced on the app layer a pre-query caching strategy:Before sending a query to Cognite, we aggregate the number of instance spaces for a given view using the endpoint /models/instances/aggregate. This result is stored in a cache layer. When a query is initiated, we check if the user included a space filter. If not, we append the known relevant spaces from the cache to the query filter. This approach has significantly reduced timeouts across our applications. However, it introduces new challenges:One request per view is still needed to fetch associated spaces. Cache invalidation must be managed periodically, especially as user capabilities may change. This workaround does not help with timeouts in the CDF UI or Infield tools, where we cannot control the q

HiI am looking into using the toolkit more actively for deploying resources to CDF. One question that was raised when researching how to use the toolkit is what kind of validations actually happens when doing a dry run for deploying data modeling resources. I do not really have any specific issue I want answered, but rather want to learn more about the tool so prepare for a lot of questions from my notes:)Does it test that the configuration of views and containers work together?Does a successful dry run mean that I can be sure that the deployment will always work?Are there anything I need to consider even after getting a successful dry run?What kind of responses do I get if the dry run finds that something is wrong? Do I get any hints about how to fix an issue?Does it consider what is already deployed into the CDF environment?Will it tell me about any issues that can happen with new breaking changes? Appreciate all kinds of insights and experiences around this topic :)Sebastian

Aker Solutions Verdal Production Line (VPL)Last week I had the chance to visit the Aker Solutions team at their impressive yard in Verdal, fabricating fit-for-purpose steel substructures and jackets for offshore developments, and a great example of industrial innovation done right.At the heart of the yard is the Verdal Production Line (VPL), a fully robotized line that opened in 2024. With automated welding, sandblasting, and painting, VPL is delivering production speeds up to 10x faster than traditional methods, cutting costs and improving safety by keeping people away from hazardous tasks.To push things even further, the team has developed their own software for weld planning. The robots scan each pipe, create a 3D model, and then autonomously plan and carry out the welding.Excited to follow VPL further. Aker Robotics 🦾Aker Solutions - Yards and Fabrication📸 Johan Arnt Nesgård 

I am able to Delete data using python code using Primary key of the table in CDF staging/RAW. But I need help with deleting data based on where condition for columns other than primary key. I am following the below documentation for deleting based on primary key. Data Ingestion — cognite-sdk 7.74.5 documentation  Delete rows from table:>>> from cognite.client import CogniteClient>>> client = CogniteClient()>>> keys_to_delete = ["k1", "k2", "k3"]>>> client.raw.rows.delete("db1", "table1", keys_to_delete)