Developer and User Community

Join the discussions about delivering apps and data-driven decisions powered by Cognite Data Fusion. Click the + CREATE TOPIC button in the menu bar to start the conversation.

  • 689 Topics
  • 2,165 Replies

689 Topics

E
Practitioner
eashwar11
asked in Developer and User Community

Timeseries upsert

How do I perform upsert using this dataframe. I also need to associate the timeseries against a specified asset. Please advise. How to use this dataframe and perform associating to an asset id using this line:    client.time_series.data.insert_dataframe(df) I have a dataframe that looks like this as below: DATE LP_Crude API_Meter LP_FRN_KERO_SW_CUTPT LP_KERO_DSL_SW_CUTPT LP_DSL_AGO_SW_CUTPT LP_MVG_HVGO_SW_CUTPT 1/1/2023 28.75472705 271.8662 440.9032 680.3416 928.875 1/2/2023 28.21111702 269.3863 466.2317 686.5167 924.1292 1/3/2023 27.78340123 268.8638 484.6189 684.5542 919.9917 1/4/2023 27.5781529 269.3117 506.7792 686.6708 921.9375 1/5/2023 28.03229217 268.1567 500.2208 688.2208 917.575

3
H
Practitioner
6 months ago
Pierre Pernot
Practitioner
Pierre PernotPractitioner
posted in Developer and User Community

How to use contextualization with the SDK

Contextualization is one of CDF’s core strengths. It allows you to link resources from different data sources together, which can be pretty powerful in an industrial context. We can for example think about linking an asset hierarchy based on SAP, to some time series extracted from sensors. You can contextualize CDF resources in the UI. In case you need more filtering capabilities, or you want to automate that, you can also do that programmatically.  In this article, we’ll take the example of contextualizing files and assets with the SDK. The setup we will use will look like that:20 assets  20 files, for which something similar to the asset name appears in a metadata field called “asset” It is often the case that files and assets don’t share something obvious to make a link on, because they might be ingested from different sources. That is where contextualization comes into play, using Machine Learning to find matches between resources based on specified fields. To create the setup, we

3
E
Seasoned Practitioner
4 months ago
R
Seasoned
Rimmi AnandSeasoned
asked in Developer and User Community

Search in JSON object type column data

I am looking for function/query that can search in  JSON type data against array of values. Json can have nested keys. I tried using JSON_SEARCH or JSON_CONTAINS . getting Undefined function:JSON_SEARCH(data->'$[*]', JSON_ARRAY("abc")) Is there a way to search in JSON object keys for list of values ? for e.g :JSON col data {"name":"Back","address":{"location":{"country":[{"city":{"state":[‘jkl’,’fgh’]},"type":"home"},"empId":"e100043867","FullName":"Back"}JSON_ARRAY = [‘name’,’state’,’country’]check if JSON_ARRAY values exists in JSON data ?Any help would be appreciated.

3
Dilini Fernando
Seasoned Practitioner
5 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

derived timeseries

I have a set of timeseries objects t1,t2,t3,t4,t5…..etc. Now I would like to create a derived timeseries objects based on the existing timeseries objects. For instance  T1 = (t1/1000) *141.5T2 = (t2/100 - t3)T3 = (t3/100 - 14)T4 = t4 * t2/100T5 = t5/100How do I perform these steps using cognite SDK. Is there a way to perform these operations?Please advise. Retrieving dataframe from data-points and then create new timeseries objects (T1,T2….) will be challenging and hence wanted if there is an optimal way to handle this in a better way. Is this possible using synthetic time-series?IS there some examples and code snippets that can be shared so that I can grasp them better?  

3
C
Seasoned Practitioner
6 months ago
R
Seasoned
Rimmi AnandSeasoned
asked in Developer and User Community

Query to get column name from different table stores as column value

We have data model withtype abc{id: Stringkind: Stringname: Stringdescription: Stringkey: String}In other table , we have these columns stored as column valueskind attributes 1.0.0 name 1.0.0 description 1.0.0 key 1.0.0 name  need query to join on column value(stored as rows) from one table with solution model table data can group on by id or kind? As there can be n number of attributes, if we can do it dynamically to match column  value with column name.Any help would be appreciated

3
Dilini Fernando
Seasoned Practitioner
5 months ago
Christian Mueller
Practitioner
Christian MuellerPractitioner
posted in Developer and User Community

The Cognite SAP Extractor is now generally available to CDF users!

Dear Cognite Hub community: We are very happy to announce that our Cognite SAP Extractor is now generally available to CDF users!The extractor connects to OData V2.0 endpoints in the SAP NetWeaver Gateway, making use of many pre-built integrations available in S/4HANA and saving considerable implementation time, while also natively connecting to the SAP data you need in a standardised manner.To get started,  download the SAP Extractor from the “Extract Data” page directly from CDF. The documentation, including Server Requirements and how to set up your SAP extractor is available here.We have also prepared a short demo (less than 4 minutes long) showing the steps needed to setup a data extraction from SAP S/4HANA OnPremise to CDF from scratch, including how to find the standard service from SAP, enable it in S/4HANA, configure and run the extractor and then see the SAP data in CDF.   Any feedback is very welcome, after testing the extractor and/or watching the quick demo please make sur

3
Christian Mueller
Practitioner
6 months ago
Andre Alves
Seasoned
Andre AlvesSeasoned
asked in Developer and User Community

PI Extractor open source, following the same approach as opcua-extractor-net.

Hi there,Are there any plans to make the PI Extractor open source, following the same approach as opcua-extractor-net?

3
Andre Alves
Seasoned
5 months ago
F
Active
fernandolaraActive
asked in Developer and User Community

Unable to connect CDF to Power BI through Streaming Power BI API

Hi all. I’ve been trying your tutorial on How to Connect CDF to Power BI, to no avail.I am not using SUbscriptions since I do not have the permissions to write one right now. So, I was just trying to execute via Time Series API or even submit a post via curl.But the problem is with the Power BI API’s URL. Eventhough I try to make a POST, by using curl, it’ll return response 401. Can anyone give me a hand here?Do I need an admin to give me write permissions?

3
Dilini Fernando
Seasoned Practitioner
3 months ago
R
Active
Rajas AthaleActive
asked in Developer and User Community

How does CDF handle Unit of measurement and internal conversions

Hi team,I have the following questions,How does CDF handle unit of measurement. Is there a concept of storage unit system and display units system Eg; If a certain time series dataset have Celsius scale and other have Fahrenheit and kelvin scale. How does CDF store these in dataset and does it offer functionality for conversion.  If I want to build a chart with multiple time series have data with different units, does CDF offer unit conversion on the fly so that I can have everything displayed in a different units? I would like to have a comprehensive documentation for it.

3
Dilini Fernando
Seasoned Practitioner
3 months ago
S
Active
sudheerchonateActive
asked in Developer and User Community

How to modify GraphQL filter using direct relation object name

I am learning GraphQL queries in Congite by following the documentation About data modeling | Cognite DocumentationHow can I add an additional filter condition based on functionalLocationParent object name value.query{listEquipment (filter: {and: [{catalogProfileDescription: {eq: "Compressors"}}]}) {  items {     externalId    name    description     catalogProfileDescription    functionalLocationParent {      name      description    }    cfihosEquipmentClass {      name      description    }  }}}

3
Dilini Fernando
Seasoned Practitioner
2 months ago
A
Practitioner
almir.mavliutovPractitioner
posted in Developer and User Community

Set parameter 'name' as required (as required in UI) for DataModelApply in cognite-sdk

1. Field ‘name’ is required during creating data model in CDF.  But it is not required if create through SDK:data_models = [DataModelApply(space="mySpace",external_id="myDataModel",version="v1")] c.data_modeling.data_models.apply(data_models) 2.  For data models have values in ‘name’ and ‘description’ fields these fields would be cleared up if these parameters are not specified in DataModelApply. These code will clear name and description values (if ‘myDataModel’ had them):data_models = [DataModelApply(space="mySpace",external_id="myDataModel",version="v1")] c.data_modeling.data_models.apply(data_models)These code won’t clear name and description values (if ‘myDataModel’ had them):data_models = [DataModelApply(space="mySpace",external_id="myDataModel",version="v1", name=”myDataModel”, description=”My Description”)]c.data_modeling.data_models.apply(data_models)To fix make changes in cognite-sdk:Set ‘name’ as required for DataModelApply. Fix clearing values from name and description in D

3
A
Practitioner
25 days ago
A
Practitioner
almir.mavliutovPractitioner
asked in Developer and User Community

Get access to GraphQL through python SDK

As Data Engineer I need access to GraphQL through python SDKto modify Data Models GraphQL: to update versions of imported views from different Data Models to read Data Models GraphQL: to parse specific info that I can not get through python SDK. For example check if view was imported what the source of imported view

3
Anita Hæhre
Seasoned Practitioner
5 months ago
Å
Active
Årne ReimeActive
asked in Developer and User Community

Data model query for Bi-directional relationships

I have a question about querying a data model using the CDF SDK.In the Cognite SDK docs there is an example for retrieving actors in a movie (https://cognite-sdk-python.readthedocs-hosted.com/en/latest/data_modeling.html#query-instances) from the following data model: But if we would change this to finding which pumps belongs to a given facility from this data model (picture below), using a bi directional relationship between facility and pump, what changes must be made to the example from the sdk docs for this to work?  

3
B
1 month ago
S
Active
Shreya PandeyActive
asked in Developer and User Community

Can I send data from blob storage to cdf raw using azure data factory?

Is there any way to send data to cdf from blob storage using azure data factory?

3
Dilini Fernando
Seasoned Practitioner
2 months ago
K
Seasoned
Kristian NymoenSeasoned
asked in Developer and User Community

Assets with GeoLocation and large polygons

Hi.We’re looking at how to work with larger geographic areas - inflow catchments for our hydropower reservoirs. The smallest one being a polygon with 3119 coordinates. Trying to create these catchments as assets with geolocation fails, due to the size limit on the GeoLocation field. Error message: “Maximum size of GeoLocation field is 10240.”I assume this means that the preferred approach is to use GeoSpatial, rather than assets with a geolocation field? If so, is the assetIds property the preferred way to connect GeoSpatial objects to assets?  And is there a simple way to retrieve the shape of an asset using the Python SDK? (something equivalent to using asset.geolocation?) Also; Any plans for showing geolocation ar geospatial information in the Fusion UI? 

3
V
Practitioner
2 months ago
K
Active
Karguvel kalisekarActive
asked in Developer and User Community

Python SDK - List, Search, Query Instance from FDM

Hi Team,Is there any sample code available for retrieving a list of instance objects from FDM? I've reviewed the documentation's sample code, but I need additional clarification on where to provide Space, Object Type, Version, and Data Model.  Sample code available for Instance List. instance_list = c.data_modeling.instances.list(limit=5) I have also followed the steps outlined in the document URL below.. advancedListInstance queryContent searchInstances Thanks,Karguvel K

3
K
Active
5 months ago
S
Committed
Shabnam OghbaieCommitted
asked in Developer and User Community

build automated pipeline to create and upload 3D model

Looking at API documentationhttps://pr-411.docs.preview.cogniteapp.com/api/v1/it seems one can not build an automated pipeline to create and update 3D model only using python SDKs?I aim to make a pipeline using cognite function and python sdk for nightly update reading files from SharePoint and upload or update 3d models in cdf.If its possible, where I can find the guiding document?

3
S
Committed
3 months ago
S
Committed
Shabnam OghbaieCommitted
asked in Developer and User Community

OPC UA extractor connection to CDF fails

I have provided the essential credentials within the config.yml file in the OpcUaExtractor config folder, however when run it get this error. How can I fix this?

3
S
Committed
3 months ago
N
Seasoned
Neerajkumar BhatewaraSchlumberger
asked in Developer and User Community

OPC UA Extractor - is there a parameter to handle the source timezone similar to DB Extractor

Hello,In DB Extractor, I see a parameter that could be configured to specify how to treat the timezone parameter.  Is there something similar for OPC-UA Extractor as well ?If not how can the different timezone data could be handled or atleast Local timezone data ?

3
E
Practitioner
5 months ago
H
Active
Hakim AREZKIActive
asked in Developer and User Community

Testing CogniteAuthError with monkeypatch_cognite_client

Hello, is it possible to test the  CogniteAuthError using monkeypatch context manager  ?Regards ,Hakim Arezki 

3
Dilini Fernando
Seasoned Practitioner
4 months ago
judekumar87
Practitioner
judekumar87Practitioner
asked in Developer and User Community

Requesting to increase CDF Time Series API limitation

It was requested by @ibrahim.alsyed  from Celanese that we increase the limit of the raw datapoints for the endpoint /timeseries/data/list (Retrieve data points) to 1 million. Currently, the non-aggregated data points returned is limited to 100000.For the drill-down views with more than 7 days, the server-to-client data transfer maximum size is reached due to the number of datapoints (some timeseries have more than 1 datapoint per minute). The solution they have implemented is that for ranges higher than 7 days, they are displaying an interpolated trend using the maximum absolute values for 30-minute aggregation. Once the user selects a smaller date range, they are unable to display all the values. Hence a increase in the limitation was requested.

3
I
Seasoned
3 months ago
N
Committed
Navyasri IndupalliCommitted
asked in Developer and User Community

Accessing Cognite Lab

I am trying to access the lab.cognite.com in my company laptop and not able to login. Is that path will not be accessible in company provided laptop rather should I use my personal laptop for lab sessions practice?Can anyone suggest on this?

3
N
Committed
3 months ago
S
Active
Shreya PandeyActive
asked in Developer and User Community

Cognite Machine learning

How can I use Machine Learning Models inside Cognite data fusion?

3
Dilini Fernando
Seasoned Practitioner
3 months ago
A
Active
aravindActive
asked in Developer and User Community

Reg. 3d model and programatic interaction

I am looking for information related to 3d models. I would like to explore does reveal/cognite supports programmatic change of animation on 3d models which are uploaded to cdf. What i am looking for is, i have 3d model of pump and would like to change the animation speed depending on the speed i get from my api (basically real time speed which i get from my api). Does it possible ? If so could you guide me to example code to how to do that. Also the uploaded 3d model in cdf is not displaying like how the original model that was uploaded. After uploading into cdf its not displaying with colors and animations, could you tell me why it is like that ? Thanks 

3
Dilini Fernando
Seasoned Practitioner
3 months ago
J
Active
JasonSianandarActive
asked in Developer and User Community

Match Time series to assets

Hi, I can’t seem to contextualize my assets to my time series in the exercise. I have followed all the steps before.  

3
Dilini Fernando
Seasoned Practitioner
1 month ago

Badge winners

Show all badges
Terms & ConditionsCookie Settings