Skip to main content

Hello, 

Thank you for all the great docs and tutorials you’ve put together on CDF.  I would like to get an expert opinion on the best way to accomplish a specific use case I have.

We have a daily time series that we calculate for predicted availability of an asset, for a time range that is usually between 5 and 40 years (usually between 1,826 and 14,610 data points per time series).  We have a view were an engineer will want to see this time series graphed, but also will want to see a single KPI value that represents the average of the time series from today to end of the range.  For all of our assets, this KPI could be ready several times a day as people view this KPI.  When we were storing this data in Azure SQL, to speed it up, we pre-calculated and stored this KPI average and stored a new time series (throwing away some of the time series points and linear interpolating).  That way, reading a single data point representing the average of from today to the end of the range and display it quickly on the screen.

 

Considering that this value is read often and needs to be displayed quickly, what is the CDF native way of making this KPI available and accurate daily?

Hi Ben

The time series database already create data point averages for for low resolution views, allowing you to quickly load and view e.g. 40 years of data suitable for e.g. a plot spanning 200 pixels on the screen. In your use case though I assume the total average is calculated every time a new data point is inserted the raw time series?

A feature to try out is the synthetic time series and it aggregation features: https://docs.cognite.com/dev/concepts/resource_types/synthetic_timeseries/

These are virtual time series acting as a function of one or more time series. You can also try this out in Charts: https://charts.az-eastus-1.cogniteapp.com/ (docs: https://docs.cognite.com/cdf/charts/#open-charts)

Regards,

Torgrim

Thank you for the info, Torgrim.  That was the other complexity of the situation I had left out.  The value not only has to change daily, but each time a new data point arrives.  Currently most of our data sources are daily or lower resolution, but this could certainly change to higher resolution in the future.

 

I had discussed with Morten and this is the additional detail he provided:

The time series capability in CDF can do this on the fly. Aggregates are automatically created by the backend so that you can in a few milliseconds get averages and other metrics. For python details, see https://cognite-docs.readthedocs-hosted.com/projects/cognite-sdk-python/en/latest/cognite.html#retrieve-datapoints

You can set aggregate to "average"

 

Great to know CDF can provide these aggregates in milliseconds and I no longer need to maintain my own code to pre-aggregate these values.

 

I am going to need to look further at synthetic time series for another case where we want to get the daily average of a time series across all direct child assets.  The tricky part I think in this other case will be specifying which time series from the child assets need to be input to the synthetic time series.  Although, I may have to rethink this as there is a limit of “100 input time series referrals” for synthetic time series.

 

Thanks!

Reply


Terms & ConditionsCookie Settings