Community Discussions

Hi There,How do you configure the state of OPC-UA extractor in the Raw layer?I know that you can do it through various types of state stores including InfluxDB or local, but I have a dataset and a table in the Staging area which I would like to get populated.Thanks in advance!

Running Cognite DB Extractor 3.0.0 as a Windows Service. Note that previous versions of this extractor have worked fine although some changes have been made to the config.yml since upgrading. Symptoms - According to the log, Upon starting the service, It seems the extractor will run and successfully query and upload (30 rows in this case) and then the service will STOP running immediately as indicated in the Windows Services app.There is an error in the log file about database/table already existing which does not make sense to me b/c usually errors will prevent the service from starting at all.  https://www.yamllint.com/ indicates the config.yml is valid. The only entry in Windows Application log is an “Informational” note that states:The description for Event ID 4098 from source DBExtractor cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer

Hi. What is the recommended method for access control of 3D models? 3D capabilities can be scoped to data sets, but how can I assign a model to a particular data set, so that it is only accessible by a limited user group? Is it possible via the Python SDK or GUI?

Hi. When setting up CDF we were recommended to use CDF data sets scoped to data sources to control access to data. However, we have use cases where we would like to share a selection of data from a source with external parties. What would be the recommended setup for this? Would we need build an API layer on top of CDF,  to copy data from a source dataset to multiple other datasets for sharing? Or could the security categories feature be used for this?The person in charge of sharing data in our case will often be a data owner / SME without much coding experience. Is there a way our data owners could move/copy a subset of data from a source dataset to a sharing dataset, or somehow label individual time series or assets in a manner making them accessible to specificaccess groups?

Hi We have a web application that uses the cognite sdk to present timeseries and datapoints.We have two use cases:1. We have a case with 500 time series, and we need to retrieve the latest 1000 data points from the combined 500 time series.2. We have a case with 500 time series, and we need to retrieve all datapoints between two dates. For both cases we also need to know which timeseries each datapoint belongs to and we need raw values and can not use aggregates.What is the most effective approach to achieve this?

I am looking for function/query that can search in  JSON type data against array of values. Json can have nested keys. I tried using JSON_SEARCH or JSON_CONTAINS . getting Undefined function:JSON_SEARCH(data->'$[*]', JSON_ARRAY("abc")) Is there a way to search in JSON object keys for list of values ? for e.g :JSON col data {"name":"Back","address":{"location":{"country":[{"city":{"state":[‘jkl’,’fgh’]},"type":"home"},"empId":"e100043867","FullName":"Back"}JSON_ARRAY = [‘name’,’state’,’country’]check if JSON_ARRAY values exists in JSON data ?Any help would be appreciated.

We have data model withtype abc{id: Stringkind: Stringname: Stringdescription: Stringkey: String}In other table , we have these columns stored as column valueskind attributes 1.0.0 name 1.0.0 description 1.0.0 key 1.0.0 name  need query to join on column value(stored as rows) from one table with solution model table data can group on by id or kind? As there can be n number of attributes, if we can do it dynamically to match column  value with column name.Any help would be appreciated

Hi I have an application to read data from and I have two options either to use ODBC or OPC UA directlyWhat are the benchmarks or limitations when it comes to aboveI have 30,000 data points to read with sample time of 1s and 5s and 10s 

Hi Team,I’m using OPC UA Extractor to extract time series data from Honeywell PHD source system.I have configured history in configuration for backfilling for 30days.But, i can see more historian data than expected please find the screenshot below.I have used following configurations in history. start-time: 0 end-time: 30d-ago max-read-length:30d  Let me if this configuration is correct for fetching 30days previous day.Please provide configuration if above one is wrong.

Hello! Is there a repository where the Aker BP open data is available for direct download? ( asset hierarchy, work orders, P&ID file)? 

Hello everyone,First off, big thanks to everyone here. You've been super helpful as I've been figuring things out with Cognite. I've hit a bit of a snag though, and I'm hoping someone here might have an answer.Check out the attached screenshot. It's got a 3D view of an asset and one of its time-series. Now, I've also got this other chart that shows the time-series for the asset, along with some predictions and models I’ve been playing with. (See Below) Here's my question: When I'm looking at the 3D view of the asset, is there a way to see this full chart with all the models and predictions instead of just one time-series? I've looked all over the forums and tried searching everywhere, but I can't find an answer. If someone could help out, that'd be awesome. Thanks so much for always being there and helping out. I'm excited to show off what I've been working on with Cognite! Cheers, Vishnu Iyengar

I’m getting this error after running the Timeseries transformation:  It is relevant to mention that the preview of the SQL query looks fine, prior tu actually running it, looks fine (data can be find here):  Many thanks in advance for your comments. Luis

what is cybersecurity or data governed for moving data from Datasource in customer site to CDF ?

 Hi,I have one basic question on Cognite connectivity with PBI. Currently, I see, I have only one option as imported connection. Is there any way we can pull LIVE data from Cognite? We would like to have real data coming from cognite connected thru Power BI  if that’s possible. Thanks in advance!

Hi. I’ve found a few references to “unit catalog”, “target unit”, and “target unit system” in the python sdk documentation. However, I wasn’t able to find a more complete description of this functionality. I assume it is there to assist with unit conversion, simliarly to what you can do in Charts? Can this also be used with custom units, that are not in the Charts standard units?Is documentation/explanation of this feature available somewhere?

Hi! Recently we received a warning from Grafana, that AngularJS-based plugins (incl. CDF data source) are going to be deprecated by the end of summer 2023 and AngularJS functionality will be removed in the Grafana v11. We tried updating to the latest version (3.1.0) but the warning is still there. https://grafana.com/docs/grafana/latest/developers/angular_deprecation/  Provided we update the plugin on time, is there still reason to worry about it? Thank you!

As Data Engineer I need access to GraphQL through python SDKto modify Data Models GraphQL: to update versions of imported views from different Data Models to read Data Models GraphQL: to parse specific info that I can not get through python SDK. For example check if view was imported what the source of imported view

Hello Community,cdf stores the data in cloud, could  anyone know what is the database / database structure being used,and any reason for using Spark SQL for transformations.

Can someone revert with proper procedure for using any of the cognite data fusion API using postman 

Hi Team, From the cdf-fundamentals training, I have mapped required details but data is displaying as a linear format not trends.   

Hi there,Are there any plans to make the PI Extractor open source, following the same approach as opcua-extractor-net?

Hi Team, I need help to write the unit test using pytest,below is the code:    row_list = cognite.raw.rows.list(file.destination.database, file.Extractor_Last_Run_Table, limit=-1,                               columns=["key", "LastUpdatedDatetime"]).to_pandas()    if not row_list.empty:        val = list(row_list[row_list.index.isin([file.Extractor_key])]['LastUpdatedDatetime'])Could you please help to write the unit test for above code?Thanks

SOLVED:The Cognite client was set up wrong, hence the Cognite API error when uploading data to CDF. See docs posted by Peter for setting up config.yaml for the extractor. Specifically these were the changes:``` COGNITE_BASE_URL=https://westeurope-1.cognitedata.com (not Fusion URL)COGNITE_PROJECT= {some project name}COGNITE_TOKEN_URL= https://login.windows.net/{some_tenant}/oauth2/v2.0/token(Different from OpenID connect token URL)```_______________________________________________Hello curious and helpful members of Cognite Hub,I’m building my first REST extractor to GET a response (sensor data) from a camera_sensor API, where I then aim to upload the data to corresponding timeseries. I have multiple cameras with each its URL path, so it seems @extractor.get_multiple(paths) gives me the desired behaviour of essentially looping @extractor.get through the URLs. However, this error arises when my extractor tries to upload datapoints to the timeseries in CDF.^CTraceback (most recent call la

Hi, I want to transform JSON object to Data model column. JSON is populated from extractor to raw table, so its type is by default Object and its structure will be nested.Is there any built-in/ Custom function using which I can parse this json  and convert the specific value into column. I have tried to_josn () which will convert the json structure to string and then with the help of get_json_object() I can get expected value by passing json path to this function. But problem here is it will always return value as String.  

Hi, I have done the Grafana setup from azure managed grafana from azure portal. But don’t have permission to install cognite data fusion in grafana from azure. Please let me know how to install plugin and what permission needed.