Synthetic Time Series : Set Points and Process Values

As in any industry, we have a lot of time series which correspond to Process Values and Set Points. I set out to do a seemingly simple task. Calculate a new time series that was 1 when the PV was +-10% of the SP and 0 otherwise. 

The first issue that I faced is that we can’t do logic calculation with Snthetic TS expressions but I won’t go into detail about that since I already made a post about it before. 

The second issue was unexpected. The Set Point temperature of an oven or other industrial material is seldom modified. For good quality, consistent set points are important. Thus, there are far fewer points in a Set-Point time series than in a Process-Value time series. 

I was trying to calculate this time series for 11h00 : 11h05.  I had a set point set to 180 DEG_C at 09h00. My process values were in the hundreds, about one every 5 seconds (or whenver the temperature changed beyond a 0.1 DEG_C threshold). With data like this, if I try calculating the expression “PV, SP*0.9, SP*1.1” between 10h00 and 11h00, it will result in NAs for the SP columns! This is because the last known value of the SP is at 09h00 and there are no known SP points between 11h00 and 11h:05. 

My workaround was to retrieve_latest for the PV and use the time stamp to write to a NEW time series with the default time_series.data.retrieve endpoint, all values within 11h00 and 11h:05 and include_outside_points in that write, and then also forward fill it so that I had SP points in between 11h00 and 11h05. 

In a prior version of my program, I deleted the new time series at the end of the program but CDF started to give me Timeout errors at the delete (Not 429, just time out). I assume I hit some kind of API limit. 

We deal with hundreds of such pairs of Set Points and Process Values. I feel that there should be a metadata field in a time series that allows it to be filled-down. That is to say, an indicator that if the last point is at 9h00 and I ask for data at 11h00, CDF can safely assume that no new points indicate no change in value, up to the datetime.now().