Skip to main content

Community Discussions

Join the conversations to shape a safer, more efficient, and sustainable industrial future!

1263 Topics
D
Active
duncan.murrayActive
asked in Product User Community
Sync API does not pull changes to data

Hi, I have been trying to use the sync API to try and pull changes from a data model - specifically node instances from a view. When testing, some fields within a view was changed / updated but I was not able to pull anything new using the sync API. This was after I created a baseline of the full dataset using the Query endpoint. The sync endpoint did work when new nodes were added to the view, it would only pull the new row, so I think my query is correct. Is there some sort of parameter to turn on / other way to use the sync API to retrieve changes and new additions to instances?

1015
Arild Eide
Seasoned Practitioner
S
Seasoned
Sonali Vishal PatilSeasoned
asked in Product User Community
What is the maximum numbers of the rows supported in cognite staging tables

What is the maximum numbers of the rows supported in cognite staging tables

41
Mithila Jayalath
Seasoned Practitioner
A
Seasoned
Ali-AkramSeasoned
posted in Developer Community
share point extractor set up

can someone please explain the this procedure for me with details ? because I got really confused. so first we register an app and create a client secret and add api read all permission n Microsoft azure, after that we create an app in SharePoint and we add the “permission request” to it later. after that what is next before downloading the extractor ? how are the two apps linked ? I saw someone at my work using Microsoft graph to link them but did not understand the logic behind it. and if this how they are linked , how ? when in the Microsoft when doing the post each app dose not mention the other

101
Mithila Jayalath
Seasoned Practitioner
Ayush Daruka
Seasoned
Ayush DarukaSeasoned
asked in Developer Community
Non-writable error while populating data into new version of a view

Hi, I have a view called "Entity" in an existing model with mandatory properties "name", "description" and "isVirtual".Now, I have a new version of "Entity" in which I have decided to implement CogniteDescribable from CDM for name and description, where these fields are not mandatory.When I try to ingest data into the new version of this view, it gives me a non-writable error since the mandatory properties that I had created for the older version is not getting populated.Is there any way to overcome this?Can I map the mandatory container to the container used in CogniteDescribable, so that when I ingest data in the new version, it automatically ingests the mandatory container as well?If yes, can you please share a YAML example of the same to be deployed through toolkit? Please note that both the mandatory and non-mandatory containers have the same name "name".

304
Ayush Daruka
Seasoned
R
Seasoned
Rimmi AnandSeasoned
posted in Product User Community
Dynamic Tasks processing in sequence

I have created 1 workflow , in which I am creating dynamic tasks depending on input, it creates batch of ids and create tasks out of it. Below is workflow definition WorkflowVersionUpsert( workflow_external_id="test_dynamic-0729", version="1", workflow_definition=WorkflowDefinitionUpsert( description="This workflow has two steps", tasks=[ WorkflowTask( external_id="test_sub_tasks", parameters=FunctionTaskParameters( external_id="test_sub_tasks", data="${workflow.input}" ), retries=1, timeout=3600, depends_on=[], on_failure = "abortWorkflow", ), WorkflowTask( external_id="test_create_sub", parameters=DynamicTaskParameters( tasks="${test_sub_tasks.output.response.tasks}" ), name="Dynamic Task", description="Executes a list of workflow tasks for subscription creation", retries=0, timeout=3600, depends_on=["test_sub_tasks"], on_failure = "abortWorkflow", ) ] ) As part of this workflow, I have some task that are needed to be executed in parallel and are expected to finish in around similar

29529
N
Seasoned
S
Active
Seerangan_UvarajActive
posted in Developer Community
CDF Learning for data engineers

I am starting the Cognite Data Engineer Learning Path. How to get the access to Cognite Data Fusion platform to practice the learning.

233
Andre Alves
MVP
P
Active
Pranjul SinghActive
asked in Developer Community
Trigger a workflow once a new instance is inserted in Data Model container

Hello experts, We want to achieve the following workflow: Once a new instance is created/added in our datamodel container Another workflow must be triggered immediately We do not want to create a scheduled based polling to invoke our workflow since our workflow is time sensitive and high importance Could you please let us know if CDF supports this functionality? Or suggest a way to achieve it. I have explored the CDF in my limited knowledge, I found that subscription is available on Data Model, but it is not enabled in our environment. Could you please help me in this. Thanks, Pranjul Singh ​​​​​​​

535
Everton Colling
Seasoned Practitioner
Chris Selph
Seasoned
Chris SelphSeasoned
asked in Product User Community
Transformation Failure

Reporting as requested. The system seems very unresponsive SQL Transformations internal error. Please report this error to Cognite. Query job ID: a3b2004a-0b5f-4e43-82bd-c396c3bf1a7d | code: 400 | X-Request-ID: 2dd10804-d4c6-9cce-b35c-a1b8af8574c1

171
Mithila Jayalath
Seasoned Practitioner
S
Committed
Shun TakaseCommitted
asked in Developer Community
Is there a way to retrieve multiple time series at once using the SDK?

When retrieving time-series data registered in CDF using the SDK, I use the following code. client.time_series.search(name="some name") In this case, if I want to take 100 time series, I would need to run the above code 100 times, but is there any SDK that allows me to retrieve this in a list format or something similar?

191
jtfrandsen
Committed
S
Seasoned
Sangavi MSeasoned
asked in Developer Community
is there a distinct filter for Data modeling sdk instance query method

I'm using python sdk and wanted to query instances based on a condition using “data_modeling.instances.query” method. i have a view “TimeseriesProperty” and it has a direct relation/reference to "TimeseriesPropertyType" view. Is it possible to have a distinct filter on the result, because multiple “TimeseriesProperty” instances can have the same "TimeseriesPropertyType" instances , we don't want duplicated "TimeseriesPropertyType" instances coming in response for below query: view_id= ViewId(space="slb-pdm-dm-governed", external_id="TimeseriesProperty",version="2_0")v_id_2_PROP = ViewId(space="slb-pdm-dm-governed", external_id="TimeseriesPropertyType",version="2_0")query= Query( with_= { "TimeseriesProperty":NodeResultSetExpression( limit=10, filter= HasData(views=[view_id])), "TimeseriesPropertyType":NodeResultSetExpression( limit=10000, direction="outwards", from_ = "TimeseriesProperty", through = view_id.as_property_ref("propertyType"), filter= HasData(views=[v_id_2_PROP])) } , sele

827
S
Seasoned
C
Active
carriechungActive
asked in Product User Community
Early Adopter of DirectQuery in Power BI

Hi, As mentioned below, which group to sign up for early adopters of DirectQuery in Power BI? https://docs.cognite.com/cdf/dashboards/guides/powerbi/dq_why I can’t find relevant group from what I see from here https://hub.cognite.com/groups Any tips ? :) Thanks,

985
I
MVP
Peter Quinn
Committed
Peter QuinnCommitted
asked in Product User Community
Is there a InfluxDB extractor?

We have customers using InfluxDB as a local on-prem storage mechanism. What’s the best extractor to use to push this data to CDF? I’ve heard OPC-UA is the way to do it, but what I find online are ways to store OPC-UA data into Influx, but not ways to get InfluxDB data into a OPC-UA server so I could use the OPC-UA extractor to get it into CDF.

1195
Taylor Zwick
Practitioner
E
Active
Ethelbert DerigeActive
asked in Product User Community
Applying textures to 3D model imports

Our team is handling 3D models in various file formats and applying colors to different models before importing them into CDF. Additionally, we are applying textures to models using Navisworks Simulate and exporting them in .nwd or .fbx formats. All related files are zipped before uploading them in CDF. According to the Cognite 3D documentation, this should be possible. However, when we import the models, the colors do not appear as expected. Could you provide guidance on the correct procedure to ensure that colors and textures are retained? Is there a specific workflow or setting we should follow?

685
G
Practitioner
Anders Brakestad
Seasoned
Anders BrakestadSeasoned
asked in Developer Community
Limitations in Incremental refresh in power BI using CDF connector

Use Case I am building a dashboard in Power BI that will visualize Events from CDF (about one million events per year). So to make the solution scale I want to use incremental refresh of the semantic model so I only refresh the model with the newest Events since the last refresh. I have followed your tutorial on Incremental refresh (which seems copy/paste of Microsoft’s tutorial), but I still have questions: What happens if the StartTime attribute of an Event is a date/time/zone when loaded into the model? The attribute needs to be a date/time type, which means I need to convert the type first. How will the incremental refresh work if the model needs to load the data in order to convert the type, and then apply the RangeStart/RangeEnd filters? Can I use Incremental refersh with the new REST API Connector? When I load events with the new connector I get the start times as number of milliseconds since epoch. So the type conversion from above also applies here. Thanks for your help! Ander

587
Anders Brakestad
Seasoned
A
Seasoned
AbdulmonimSeasoned
asked in Developer Community
Pygen - Filtering data record column

I have a data model and would like to retrieve or list nodes from a specific view. Specifically, I want to list nodes that have been created or updated, for instance in the last 30 days, using Pygen. I understand that Pygen provides filtering capabilities to filter based on view properties or columns. However, I don't see a way to filter by the updated date, which is a column automatically created under the data_record column when inserting new nodes with Pygen. It seems Pygen doesn’t expose this field as a filter parameter. Is there a way to apply a filter on data_record.updated_date using Pygen?

180
A
Seasoned
Everton Colling
Seasoned Practitioner
Everton CollingSeasoned Practitioner
posted in Product User Community
Microsoft-Certified Power BI REST Connector Now Available!

The new CDF Power BI REST connector has been certified by Microsoft and is now included in the latest Power BI desktop version and deployed in the Power BI online service.What's new with the Power BI REST connector: Flexible authentication: Connect Power BI with any IdP supported by CDF (the legacy OData connector only worked with Azure Entra ID) Broader data access: Fetch data from OData services (just like the legacy connector) Access data from Data Models using GraphQL Connect to any GA CDF API endpoint Significant performance boost: Up to 10x faster when using regular REST endpoints compared to fetching the same data via OData The connector is currently in Beta, and we're eager to hear customer feedback before promoting it to GA. The documentation for the new connector is available here, and we're working on a new set of micro learning modules in Academy based on the new connector.

11010
Andre Alves
MVP
A
Committed
Alex NarayananCommitted
asked in Product User Community
Support for DFOS (Distributed Fiber Optic Sensing data

Hi Just wondering if CDF extractors/connectors has a support to fetch DFOS data, say for example to measure temperature along the line. Access may perhaps be through the third party SDK or a native connector. Also interesting would be how such high frequency data can be modelled in CDF core and solution models correctly. Any pointers would be of great help! Regards Alex

111
T
Seasoned Practitioner
Chris Selph
Seasoned
Chris SelphSeasoned
asked in Product User Community
Canvas Data Grid Issues

The data grid object has some issues when changing the grid like removing a row or adding a column etc. The attached video shows the behavior. A browser refresh clears up the “jitter” but changes do not take effect.

250
Chris Selph
Seasoned
S
Committed
ShwetaCommitted
asked in Developer Community
Cognite functions

Hi team, I want to import some functions from util folder which is same directory of the folder in which handler.py. As files in util folder is being used by other cognite functions too, I can’t keep the files of util folder in function_folder but at the time of importing functions from files of util folder and after deploying cognite function it gives import error as so basically structure is Repofunction_folder- file1 and handler.pyutil_folder - file2 and file 3 And I want to use methods from file 2 into handler.py and I have added __init__.py in util folder 2025-04-04T12:17+05:30 : Traceback (most recent call last): 2025-04-04T12:17+05:30 : File "/home/site/wwwroot/function/_cognite_function_entry_point.py", line 297, in import_user_module 2025-04-04T12:17+05:30 : handler = importlib.import_module(handler_module_path) 2025-04-04T12:17+05:30 : File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module 2025-04-04T12:17+05:30 : return _bootstrap._gcd_import(name[

181
H
Practitioner
C
Active
catherinebennettActive
asked in Developer Community
Handling Property Schema Evolution in CDF Without Breaking Existing Data Models

Hello As our industrial data evolves; we're facing challenges with schema versioning specifically when modifying / extending the properties of existing asset or time series types in Cognite Data Fusion (CDF). 🙂For instance; adding new fields or changing data types (e.g., integer to float) in metadata often breaks downstream pipelines or applications that expect the original structure. CDF doesn’t enforce a strict schema, but how do others handle backward compatibility in real-world use?😐 We’ve experimented with versioned types and custom labels to signal changes, but this quickly becomes hard to manage at scale. Some of our consumers rely on fixed field names, and introducing Salesforce Developer Course breaking changes results in unexpected behaviors in the SDKs or fusion apps.😐 Is there a recommended practice for managing schema evolution that preserves data integrity while enabling flexibility for growth?🤔 Looking to hear how other developers are addressing this challenge—do you

100
C
Active
M
Seasoned
Marwen TALEBSeasoned
asked in Developer Community
Where is pygen.graphql_query() method?

Hello, As per the docuementaiton Pygen exposes a `graphql_query` method. However, I’m not able to find it. Could you please help? version used: cognite-pygen==1.2.1 Thanks

41
Anders Albert
Seasoned Practitioner
M
Seasoned
Marwen TALEBSeasoned
asked in Product User Community
Access granularity per space on data models level is not working

Hello, While testing data gouvernance feature per space in the data model service, we noticed that it does work on the instance level but not on the data models level. Here are the tests conducted: Test 1: We tested giving access to users to only instances that are in a specific space but applying the following ACL : - dataModelInstancesAcl: actions: - READ scope: spaceIdScope: { spaceIds: ['test'] } When requesting the data model, only instances in the test space are returned. ACL works on instance level. Perfect! Test 2: When applying a similar ACL on the data model level : - dataModelsAcl: actions: - READ scope: spaceIdScope: { spaceIds: ['not_that_important_data_models'] } In this case, we notice that we get revoked access to all data models, even data model in not_that_important_data_models space. We expected that we would be able to access only data models in not_that_important_data_models. Could you please check ? Thanks

141
M
Seasoned
J
Active
Jan DolejsiActive
asked in Developer Community
Cognite Data Fusion Python SDK - how to delete edges that lead to a node with a specific target_edge_id?

I am using the [Cognite Data Fusion Python SDK](https://cognite-sdk-python.readthedocs-hosted.com/en/latest/data_modeling.html#delete-instances). While deleting a node, e.g.... ```pythonclient.data_modeling.instances.delete(NodeId(space='some-space', external_id=deleted_node_id))``` I want to also delete the edge(s) from the parent in order to avoid leaving orphan edges in the database. **Problem**: how can I look-up such edges? I know how to get all edges (and then filter by the `end_node` attribute), but that would be very inefficient: ```pythonedges = client.data_modeling.instances.list( instance_type="edge", limit=-1)``` How could I construct the `Filter` for the `end_node` (to match the `deleted_node_id`) and ideally the `type` (to equal `Child` type)? E.g. ```pythonedges = client.data_modeling.instances.list( instance_type="edge", filter=And(Equal(???), Equal(???)), limit=-1)``` Because then I could just delete all those edges with one `client.data_modeling.instances.delete(edges

470
J
Active
E
Active
Erik WikerActive
asked in Product User Community
Cascading delete of instance

Hi! Is there a way to enable cascading deletes for instances for both direct relations and edges? Edit: I posted in the wrong group, but i don’t think i can delete.

513
J
Active
Maria Fonseca
Seasoned Practitioner
Maria FonsecaSeasoned Practitioner
posted in Open Industrial Data
How to generate a client secret for Open Industrial Data?

In Open Industrial Data (OID), we have moved away from API keys. Open Industrial Data currently supports Open ID connect. You can check out more on how to configure OpenID Connect on Open Industrial Data. If you are planning to use OID and authenticate using client credentials flow, you will need a client secret from the app registration on the Azure Active Directory. Go to Open Industrial Data and you will observe there is a widget for generating a new client secret ID In the drop down, you can select two options: Other: Use this if you are using Postman or Python SDK Javascript: Select this option if your app is in Javascript Once you click on Create client secret, this will be display just once. Make sure to save it somewhere safe. Let me know if you have any questions 🙂

118213
C
Active
Terms & ConditionsCookie Settings

Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings