Community Discussions

Hi! We’re aiming to monitor all of our CDF operations in the solution we have for monitoring other systems and data flows (Splunk). This would involve monitoring of extractors/extraction pipelines, cognite functions, and general platform availability. I wonder if any other Cognite customers have done this, and if you would be willing to share your experience / architecture for extracting logs from cognite functions etc. into Splunk or a similar service? Cheers,Kristian 

When we are selecting different date range, the granularity is changing for different number of days selection. For 2 days - 3min; For 3 days - 5min.Can we get access to any document how this pattern is designed. 

I am trying to create a synthetic time series query that looks are two other time series and if those values meet a certain condition, is outputs a 1 or 0 as the synthetic time series but I keep getting a syntax error that I can’t solve. Below is a code sample. temp_id_list = [3308928194658464]flow_id_list = [1256034258207391, 3564715377369546]for temperature in temp_id_list:    temp_expression = f"TS{{id:{temperature}}} < 200"    flow_expression = " + ".join([f"(TS{{id:{flowrate}}})" for flowrate in flow_id_list])    final_expression = f"if({temp_expression} and {flow_expression} > 0, 1, 0)"    cv_query = client.time_series.data.synthetic.query(expressions=final_expression, start="1d-ago", end="now")

I am trying to read timeseries data from CDF on my local but facing SSL Error also showing cognite.client.exceptions.CogniteConnectionError.[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:997)  

I’m getting results from several topics using the # function at the end of my topic path-tree.e.g. ‘~/Installations/Location/Vessel/Seapath/#’where ‘~/Installations/Location/Vessel/Seapath/acceleration_x’ is an example pathThe problem now is that: when I get “context.topic” I get the full path (~/Installations/Location/Vessel/Seapath/acceleration_x) instead of the last path (acceleration_x) and I can’t seem to find a way to extract only the last path as there is barely any functionality in Functions | Cognite Documentation.Is there a way to get past this?sample data: {"timestamp":1700740159863,"values":[-0.013358482159674168]}here is my transformation:{  "externalId": concat("installation.vessel.seapath.", context.topic_last_part), # where I can e.g. do split,

Hi,From my MQTT topic I receive sensor data as a timestamp and 25 values, where each value should gets transformed as a datapoint for its own timeseries. However, there is a limit on 2000 char for the transformation code AND a constraint that does not allow you to set up duplicate streams on the same topic: meaning you can only do one transformation per topic, but this function is limited to 2000 chars. I run into a problem now where, for this sample data:“{"timestamp":1701093489556,"values":[-0.035999998450279236,-0.10400000214576721,-0.09099999815225601,-0.08299999684095383,-0.09399999678134918,-0.10999999940395355,-0.10700000077486038,-0.10499999672174454,-0.11299999803304672,-0.12099999934434891,-0.13899999856948853,-0.1340000033378601,-0.14399999380111694,-0.1340000033378601,-0.15000000596046448,-0.15000000596046448,-0.17000000178813934,-0.16699999570846558,-0.15399999916553497,-0.16500000655651093,-0.15000000596046448,-0.10199999809265137,-0.111000001

Hello,In DB Extractor, I see a parameter that could be configured to specify how to treat the timezone parameter.  Is there something similar for OPC-UA Extractor as well ?If not how can the different timezone data could be handled or atleast Local timezone data ?

While testing the hosted MQTT extractor and the pre-release front end I have noted a couple of things you might be interested in, but I’m also guessing some of these are quite obvious to you and already in the pipeline. Front end should support creating a new destination with specified dataset. Viewing and editing a stream transformation without deleting and creating a new one would be nice. A way of listing existing transformation mappings and create or draft new ones without setting up an extractor instance for it would be helpful. Transform error-messages might not point at correct line and character in the message it returns. I don't have an example of this right now, I will post one if I see it again. Transformation Mappings entity could have a description-attribute that lets users describe the scenario it is written created for. The endpoint listed in beta docs for creating destinations did not work, I

Within custom transformations, is there a way of updating ongoing events in cases where the MQTT messages does not contain an UUID or enough information to build an event-externalId?In the example messages below I would ideally like to create an event based off the first message, and then update it when the second message is received. Is it possible to do so without an UUID that is contained within both messages? Events start message:{    "alarm": {        "data": {            "Name": "logic_cond",            "Source": "logic",        },        "id": "logic",        "message": "TOOL UNCLAMPED",        "state": "active",        "timestamp": "2023-11-08T14:16:41.626645Z"    },    "source": {        "applicationType": "mtc",        "clientID": "asset5",        "ownerID": ""    }}Event end

Trying to download the openapi spec over at https://api-docs.cognite.com/20230101/ by using the URL in the “download” link fails with 404 when trying to use the URL with wget/curl and “ADD” in a dockerfile. It seems like the link is generated behind the scenes during pageload by javascript. Is there a way to download the openapi spec in scripts e.g for use in CI pipelines and similar?

Hi Team,Is there any sample code available for retrieving a list of instance objects from FDM? I've reviewed the documentation's sample code, but I need additional clarification on where to provide Space, Object Type, Version, and Data Model.  Sample code available for Instance List. instance_list = c.data_modeling.instances.list(limit=5) I have also followed the steps outlined in the document URL below.. advancedListInstance queryContent 

Hello, I am going through Cognite Data Fusion Fundamentals and am having an issue that I cant figure out. After multiple attempts I am loading 1 row of data to try and figure out my issue.I make it to the “Preview the Transformation” step:  The next step is to “Run the Transformation” step and this is where I am running into an issue. I get the below error message and theres no suggestion on what to do to correct my apparent error. Any idea what to do or how to progress? I just want to finish this course. -George

Hi I have an application to read data from and I have two options either to use ODBC or OPC UA directlyWhat are the benchmarks or limitations when it comes to aboveI have 30,000 data points to read with sample time of 1s and 5s and 10s 

How to upload excel files to CDF files, I am looking for ways to upload files programatically and schedule this using cron expression.  

We’re attempting to execute 10 concurrent jobs, and each one will create approximately 20 Timeseries objects, immediately ingesting datapoints for these newly created Timeseries objects in a loop.However, if we try to run 500 similar jobs with 10 concurrently running at a time, about 10% of the jobs fail due to a ‘Timeseries not found’ error.We theorize that the creation of a Timeseries object can take some time. Therefore, immediately ingesting the datapoint might fail because Cognite doesn’t refresh quickly enough to find the corresponding Timeseries object. This issue only occurs when we attempt to run multiple concurrent jobs.Has anyone else encountered similar issues? 

Hi, we are using the PI Extractor with Extraction Pipeline and is pushing the config through the pipeline.But we noticed that after updating Config in extraction pipeline we have to restart the PI Extractor  to force it to use the updated config.If we update the config we can se in run history that the extractor is reporting “Seen” on the new config revision but it is not loading and using the new config.Is there a way to force it to use new config?

Hello,I have been trying to load a CSV file into a table while going through the Cognite Data Fusion Fundamentals course. I have tried for over an hour to load the data correctly. One of the column headers adds 2 commas after the text and is throwing off the SQL formula. I am using the stock CSV file provided in the course, loading it straight from my desktop and not making any changes to the file itself.   

Hi There,How do you configure the state of OPC-UA extractor in the Raw layer?I know that you can do it through various types of state stores including InfluxDB or local, but I have a dataset and a table in the Staging area which I would like to get populated.Thanks in advance!

Running Cognite DB Extractor 3.0.0 as a Windows Service. Note that previous versions of this extractor have worked fine although some changes have been made to the config.yml since upgrading. Symptoms - According to the log, Upon starting the service, It seems the extractor will run and successfully query and upload (30 rows in this case) and then the service will STOP running immediately as indicated in the Windows Services app.There is an error in the log file about database/table already existing which does not make sense to me b/c usually errors will prevent the service from starting at all.  https://www.yamllint.com/ indicates the config.yml is valid. The only entry in Windows Application log is an “Informational” note that states:The description for Event ID 4098 from source DBExtractor cannot be found. Either the component that

Happy to share with the Digitalization community some security news!Since Cognite's start, we have prioritized security and take our security role very seriously as a partner to mission-critical industries. Today, I am happy to announce that Cognite has obtained our SOC 2 Type II certification. The American Institute of Certified Public Accountants (AICPA) bases SOC 2 engagements on its Trust Service Criteria. These criteria ensure we have rigorous security guidelines and stick with them. Type II certifications, in particular, are comprehensive and involve significant work to prepare, so a huge thank you to our team and our partners for their contribution to this achievement. At Cognite, we are investing in more automated ways to support continuous compliance monitoring.  We fo

Hi. What is the recommended method for access control of 3D models? 3D capabilities can be scoped to data sets, but how can I assign a model to a particular data set, so that it is only accessible by a limited user group? Is it possible via the Python SDK or GUI?

Hi. When setting up CDF we were recommended to use CDF data sets scoped to data sources to control access to data. However, we have use cases where we would like to share a selection of data from a source with external parties. What would be the recommended setup for this? Would we need build an API layer on top of CDF,  to copy data from a source dataset to multiple other datasets for sharing? Or could the security categories feature be used for this?The person in charge of sharing data in our case will often be a data owner / SME without much coding experience. Is there a way our data owners could move/copy a subset of data from a source dataset to a sharing dataset, or somehow label individual time series or assets in a manner making them accessible to specificaccess groups?

Hi We have a web application that uses the cognite sdk to present timeseries and datapoints.We have two use cases:1. We have a case with 500 time series, and we need to retrieve the latest 1000 data points from the combined 500 time series.2. We have a case with 500 time series, and we need to retrieve all datapoints between two dates. For both cases we also need to know which timeseries each datapoint belongs to and we need raw values and can not use aggregates.What is the most effective approach to achieve this?

I am looking for function/query that can search in  JSON type data against array of values. Json can have nested keys. I tried using JSON_SEARCH or JSON_CONTAINS . getting Undefined function:JSON_SEARCH(data->'$[*]', JSON_ARRAY("abc")) Is there a way to search in JSON object keys for list of values ? for e.g :JSON col data {"name":"Back","address":{"location":{"country":[{"city":{"state":[‘jkl’,’fgh’]},"type":"home"},"empId":"e100043867","FullName":"Back"}JSON_ARRAY = [‘name’,’state’,’country’]check if JSON_ARRAY values exists in JSON data ?Any help would be appreciated.

We have data model withtype abc{id: Stringkind: Stringname: Stringdescription: Stringkey: String}In other table , we have these columns stored as column valueskind attributes 1.0.0 name 1.0.0 description 1.0.0 key 1.0.0 name  need query to join on column value(stored as rows) from one table with solution model table data can group on by id or kind? As there can be n number of attributes, if we can do it dynamically to match column  value with column name.Any help would be appreciated