Impact 2024: The Industrial Data and AI Conference for and by Users | Nominate Speakers Now for a Ch...
OPC-UA can carry various data “types”, and depending on which type of data you are sourcing via OPC-UA, you may want to leverage different resource types / capabilities in CDF.In simplified terms, we can make the following, high-level guidance:Structural entity data. This part of the data typically represents the setup of the control system (from which the OPC-UA data is sourced). You could think of this as the equipment and sensor breakdown structure which helps understand the context of sensors and events. CDF Data Modeling is a good destination for this part of the data. Data points, sensor values, variables to be tracked over time. This part of the data matches very well with the CDF Timeseries resource type. Events. This part of the data matches well with CDF Events (primarily), or Data Modeling (secondarily).
Hi Anders,The best way to filter/retrieve the particular type of weather data depends on how it is represented in CDF. In general, it is common to encode data features like weather type (and other descriptors) as metadata and/or labels on the time series. In this particular case, however, we recommend that you get in touch with the Miros weather data owner at AkerBP as they have the best insight to how this data is represented in your case. For example, you could try contacting the AkerBP CDF Operations team. Cheers,Kjetil
Hi. Two ways you could go about this: 1) easy and 2) hard :). The community can probaly come up with other ideas as well. Starting with 1). The core types in open lineage, dataset, job and run map very well with the CDF resource types/api endpoints “data set”, “extraction pipeline” and “extraction pipeline run”. The reason I list this as “simple” is because in this case you have extraction pipelines (jobs) feeding data to data sets (dataset)--there is no multi-step DAG. This model will give you nice, basic lineage + you can use CDFs UI and built-in monitoring and alerting. And, it is all in v1. I recommend that you investigate if this can fit your needs. More information here: https://docs.cognite.com/cdf/integration/guides/interfaces/about_integrations.html The more complex alternative, 2), is to use CDFs data resource types to represent you DAG/network. Open lineage’s “dataset” can be represented as CDF assets, the “job” is also an asset, with CDF relationships linking them into a DA
Hi Niros.We don’t offer native data push/stream capabilities in CDF yet. It is a capability that we have on our radar, but it is not a part of our (short to medium term) road map. So you have to look at some variation of setting up an agent that polls data from CDF, keep a watermark, and push to your destination. When polling data from CDF, it is worth noting the following:The time series API is eventually consistent. That is, there may be a small delay from a data point is published to CDF until is queriable. Also, CDF does not guarantee that the data points become available in a sorted order. The consequence of this is that you probably want to let the data settle for a few seconds before you query for it. That is, you client should implement a “polling offset” where you query for a time window “t - <polling offset>”. If your data source publish historic data points then CDF does not have a good way of communicating that to you--there is no “last updated time” on the data point
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.
Sorry, our virus scanner detected that this file isn't safe to download.
Check the
documentation
Ask the
Community
Take a look
at
Academy
Cognite
Status
Page
Contact
Cognite Support
