Cognite Data Fusion: Missing values in a Time Series is reflected in Synthetic Time Series

@Andreea Pastinaru @Hunter Beck what is the status of this one?

Hi Marcela,

This one is with me as the PM for Time Series.  We have begun work on the API design to implement support for Status Codes, using the OPC-UA standard.

This will allow data points to be marked as Good, Bad or Uncertain.

As part of this work, we will also be changing the behaviour of our extractors, so instead of dropping bad data points, we will instead store them with their appropriate status code and we will offer a choice of pre-calculated aggregates based on what to do with these status codes.

It will also be possible to choose not to use Bad data points (or Bad and Uncertain) in the generation of a Synthetic Time Series.

We anticipate that we will have a beta release in the early part of Q1 2024 for this.  I’d be happy to include you in the development feedback process if you are interested to participate?

Kind Regards, Glen

Hi @Glen Sykes ,

Would this new API feature help solving issues with synthetic timeseries where some datapoints are missing in one or more timeseries, as described in the source? 

Hi @Marie Solvik Lepoutre,

So with the new feature, there will no longer be gaps in the time series where the extractor drops bad data points from the source.  Instead, there will be legitimate values, but with a ‘bad’ status indicator.   

So the request here is:  If summing two time series where some data points are bad, you are asking for us to treat the bad data point as if it were ‘zero’, rather than ‘not a number’, and so the synthetic time series would add ‘zero’ to the corresponding data point and produce a number, rather than an ‘undefined’ result.

Have I stated the request correctly?

 

@Glen Sykes please keep myself as well as the following informed: @crgomez13 , @ibrahim.alsyed , and @rsiddha 

This would help some issues we’ve seen users have with calculations that pull from sources with bad indicators - their assumption would have been that it is treated as zero rather than creating an error message and not computing or having to get rid of the bad source altogether.

Hi Marcela,

I will send you a direct message as I would like to share our draft API specification with you for feedback.

Kind Regards, Glen

​@Glen Sykes Are there any updates on this? 

As Ola is writing here, we would also like to be able to do a calculation on multiple timeseries where some are not overlapping by padding the timeseries with 0-values. I’m not sure if this feature request will solve that, or if you have another feature request for padding?