Community Discussions

Hi Team, we have data in format like list of externalIds and space as key and value.  Like list of Json object in data model. something like [{externalId:well,space:slb},{externalId:well1,space:slb2}]So can we keep a field in model like violatedEntities: [JsonObject]Is it possible to parse all the externalIds and store it like this?Thank you.

We are now planning to integrate a large amount of historical data. The context about historical data and our data organization is in this topic:We create an interface WellState, which contains 2 properties: WellId and Archived, in order to propagate the Archived flag to every object in our data models.After integrated large historical data in our development env, we found that DataFetchingException always occurred in Daily(550K lines) and Operation(2M lines) in using GraphQL. And if using cdf_nodes to load Daily and Operation, the transformation will be blocked util that your session is closed.Info: Daily has many relationships and Operation has only a relationship with Daily. But We have also other 3 objects like Operation(big dataset and a relationship with Daily), they can work well. GraphQL:query MyQuery { listOperation { edges { node { externalId space } } } listDaily { edges { node { externalId space } } }}the result is alwa

Hello, I have two questions:When using cdf_nodes in transformation, does query optimizer take advantage of the container indexes? What we notice is it is not the case.  How can we run broadcast joins in transformation. Hints does seem to be working: https://spark.apache.org/docs/latest/sql-ref-syntax-qry-select-hints.htmlThank you!

WITH archived_wells AS ( SELECT DISTINCT ac.WellId FROM cdf_nodes( 'sp_dm_dap_knowledge_graph', 'Activity', '0.71' ) ac JOIN contextualization.dap_wellview_wellid_space_mapping map ON ac.WellId = map.key WHERE map.Archived = TRUE AND ac.Archived = FALSE)SELECT s.externalId, s.space, s.WellId, true as ArchivedFROM cdf_nodes( 'sp_dm_dap_knowledge_graph', 'WellState', '0.71' ) sJOIN archived_wells aw ON s.WellId = aw.WellIdWHERE s.Archived = FALSE AND EXISTS ( SELECT 1 FROM archived_wells )WellState has 20M rows of data. To improve SQL performance, I want to use a CTE to get the IDs of archived wells and then update WellState (archived and WellId are indexed). However, I found that even when using EXISTS to check that the CTE result is empty, cdf_nodes still loads instances. In this case, shouldn’t the result just be empty?Additionally, when I replaced the CTE with a subquery,

I am facing issue of jupyter notebook getting crashed frequently. It is observed for larger data and surprisingly ML model training on smallest data chunks as well.Sharing the image for reference.  

Hello,We are querying the view WfAlarms (structure as given below) which has direct reference to alarmCode. When we are querying it for say 3+ months, we get the CogniteApiError, but works fine when it is 1month. This error is seen in our application deployed in production. We expect almost 2 million of records in this view. Can you help to understand what can be done to fix this issue ? Any configuration that needs to be changed. Let me know if you need any other details.1. Properties being queried from view - WfAlarms ErrorView:type WfAlarms @view(version: "2") @container(indexes: [{identifier : "entityName", fields : ["entityName"], cursorable : true}, {identifier : "entityExternalId", fields : ["entityExternalId"], cursorable : true}, {identifier : "alarmFlags", fields : ["alarmFlags"], cursorable : true}, {identifier : "entityType", fields : ["entityType"], cursorable : true}, {identifier : "alarmCode", fields : ["alarmCode"], cursorable : true}, {identifier : "startDateTime", fie

Hi all, looking for guidance on two issues we hit with CDF related to timeseries visibility and datamodel queries.403 error leaking timeseries externalIds:when a user without access requests a timeseries resource, a 403 error is returned as expected, but the error body contains the externalIds of the timeseries. Those externalIds are sensitive and should not be exposed to users who don’t have access.Is it expected behavior that externalIds are included in 403 error messages? If not, is there a setting / configuration / log-level that controls whether identifiers are returned in errors? Any recommended mitigation or planned fix? Datamodel query returns datapoints from unauthorized timeseriesSetup: Created dataset_1 and dataset_2 Created two timeseries, TS_A in dataset_1 and TS_B in dataset_2 Attached both TS_A and TS_B to a datamodel Granted the user only timeseries:READ scope:dataset_2 (so they should see TS_B only) Observed behavior: querying the datamodel returns datapoints for T

Hi, Data models names are limited to 37 chars today. Is this a hard limit? Is there a way to extend this so we can have more possibilities for internal naming conventions? Thank you!

Can’t deselect tag when only 1 tag is selected. Pressing “Clear” will reset all filters and take me out of the menu.  Markus PettersenAker BP - Data Platform Architect

Hi team,I am looking for a way to trace data lineage in CDF. For example, if we have a table in RAW that feeds another table through a Transformation, is there a straightforward way to see where that data is being used downstream?Basically, I want to understand how to track which tables, transformations, or data products depend on a particular RAW table. Any native feature or recommended approach for this in CDF?Thanks!

Hello, We are using cdf toolkit to deploy our data models. We have a relatively a big data models that uses 478 views. Each time we deploy a breaking change our data model architecture (well centric) obliges us to redeploy a new version for all views. We notice a huge deployment time at the dependencies resolution step: 2025-08-19T09:34:54+00:00 WARNING [LOW]: Failed to create 478 views: One or moreviews do not exist: 'sp_dm_mud_brines_for_ops_real_wview:Activity/0.10'.. Attempting to recover...The overall deployment time takes 35 mins, this way bigger than our standard CI/CD job duration where we try to keep it under 10min at worst. Is there a way to optimize this please?  Thank you!    

I created an asset hierarchy with the Cogntie Core Data Model, then I created a custom data model through neat and I need to migrate the assets from the core data model to the custom extended data model, I tried creating a transformation but got the following error: failed with status 403: Properties [assetHierarchy_path, assetHierarchy_path_last_updated_time, assetHierarchy_root] are maintained by DMS and cannot be modified by end users. is there a way I can migrate/update the core data model assets to our extended data model?

Hi,Context:We currently organize our data in CDF using a per-country partitioning strategy, where each country has its own space.This approach was chosen primarily to restrict data access by country in a fine-grained manner.On top of that, we expose data models grouped by Business Object, such as Well Architecture, Cost Model, etc.Each Business Object model aggregates several data objects under a common business theme, which also allows us to control access by business domain in addition to country-based access.We are now planning to integrate a large amount of historical data, which will likely increase our model size by around 4x.These historical datasets are rarely queried, but we want to make sure their addition does not degrade performance for operational data — both in query latency and data ingestion throughput.We are evaluating two potential strategies: Keep everything in the same spaces, adding an indexed attribute (e.g., is_legacy = true) to distinguish legacy records. Crea

While I try to insert a Stream (using the http post as shown in the Getting Started guide), I am getting the next error:cognite.client.exceptions.CogniteAPIError: Project 2982735218914002 not enabled for Industrial Log Analytics | code: 400 | X-Request-ID: 66573df8-87ce-9b37-a837-180aee796a6c | cluster: westeurope-1 | project: slb-mmv-ops-dev This detail isn’t mentioned anywhere on the documentation. I cannot find how to change that. Can you enable ILA for https://delfi-dev.fusion.cognite.com/slb-mmv-ops-dev/?

I am stumbling through the Beta documentation. When I try to run the http post to create a Stream (as described here), I get this error:The feature can only be used with [alpha, beta] headersI found documentation that seemed relevant, but it seems the format of the header is this cdf-version: 20230101-beta https://api-docs.cognite.com/20230101/#section/API-versions/Beta-versionsHow do I know what date should I put into the header to gain access to the Streams beta feature?

Hi team,  do you have 2-3 articles on how to get started, like a simple quick start example as a starting point to be acquainted with the service ?Thanks !FYI ​@Marwen TALEB 

Hello,I’m using a Cognite MQTT Extractor.I wonder if it is possible to publish with this client to the same broker I am subscribed to.Furthermore, I would like to know if it is possible to develop callbacks in a similar way to Python clients, for example : https://pypi.org/project/paho-mqtt/#callbacks Regards,Pierre

Hi, I’m using the hosted REST extractor to populate a data model from an MES system. One of the API calls I need to make uses a URL parameter “id” to get data for production jobs by their id like: GET /scheduling/v1/jobs/{id}/operations . The production jobs are already in my data model and each has a unique id value.Is there a way to query the job ids from my data model within the hosted REST extractor so I can make a separate REST call for each job id? I was thinking the pagination feature would be the best approach, but I’m not sure how to query my data model from there to get a list of job ids to iterate through.Any insight is appreciated. Thanks!

Error when executed dbextractor.exe config.yml commandError when start the Cognite DB Extractor 3.9.0 Services 

Is there a way to delete the relationship between an instance and a container? Let’s say I have a view Foo in one data model and another view FooExtension in another data model. FooExtension implements Foo.I have an instance of externalId “FOO”, space: “FOO-DAT” that have data stored in both views/containers.If it was no longer necessary to store data in container FooExtension is there a way to just delete this relationship in a way that `hasData(FooExtension)` from the DMS Query doesn’t find this instance anymore without affecting the data stored in Foo and the instance itself? Since instance count is limited, it’s becoming more common in our project modeling things having the mindset of shared instance when they mean the same thing across data models but with different formats to present the data on different Apps. But sometimes we don’t want to have the data in the app anymore and virtual deletion (boolean flags) is not always an option specially when the data volume is big. 

Hello, We are a heavy users of Grafana CDF datasource. Are you planning to support alerting anytime soon?  Thanks !

 Below I have attached three screeshots where I query same data in same time range. I have used Queries in first screenshot, and it corresponds to original data inspected in Charts, i.e. there is no data before 4:01. This is very basic query using external id. However in many cases we need custome query where we use parametrised syntax. I have used a very basic syntax in the second screenshot that only use external id to query same timeseries as first screenshot. But the result is clearly different, there are datapoints created where there should be no data. Inspecting the datapoints in query, I see real data points with some values are created in no data area! Disabling aggregate resolve problem of creating non exiting data in third screenshot but then it query all datapoints and saturates in 2 seconds (reach max datapoints that grafana can show). So I need aggregated data and I have tried aggregate inside the syntax and same problem in scheenshot2 persists in this case. Any idea how

I checked the Data Management UI in CDF Fusion. There is a list of data models in the PublicData project.There, I found the Cognite Core Data Model, where the schema is well-defined, but the instance has no data.I want to explore a GraphQL query using this data model—for assets with their hierarchy and related entities such as time series and files.However, since there is no data, I am not sure how to proceed. Could you please guide me on how to solve this?if we need transform data we need permission for the write  

Hi Canvas Team -  Do we have the ability to make a public canvas that has the ability for the owners to lock & unlock it (making it essentially read only)?  Examples of this will include the ability to save a work package, turnaround package, reliability root-cause analysis, etx that others can see in the future however they can’t edit. The owners (people specifically shared) have the ability to unlock and edit it.   Other scenarios are a read-only version of the canvas for other people that are not the owner. 

We have outage on Cognite function deployments impacting both Development and UAT environment at PSaas cluster at ADNOC environment.Please refer the below url for the respective env.https://adnoc.fusion.cognite.com/sub-cgnt-panorama-dev/https://adnoc.fusion.cognite.com/sub-cgnt-panorama-uat/This is impacting on our development and UAT testing.kindly do needful asap.Thanks,Palaksha