Skip to main content

Sequences vs Events vs FDM

  • 12 June 2024
  • 3 replies
  • 30 views

Hello,

I am trying to understand the difference between Sequences and Events to see what is pertinent for me in a specific use case. For me, both of them allow saving a native CDF object along with a timestamp, enriched with other data. I feel like we can also achieve the same goal by making a data model with timestamps in one column and all the properties we are interested in, in other columns;  though this approach would be harder because it would necessitate using pygen for input output. 

In my company, we work on Industrial processing lines. At any given moment, a line is operating under a set of conditions, defined by a fixed set of set points, like ‘conveyor speed’ and various temperatures. Let us call the set points a, b, c etc. For set points a, b and c, we have measured values i, j and k.

To assess whether the machine is operating normally, we want to compare the aggregated measurements (i-j-k) taken during a period when the set points are a-b-c with a previous instance when the same set points occurred.

If ijk are very different from other ijk-s for the same abc, we can alert the operator that something isn’t right. 

In other words, we want to classify ranges of time as ‘Batches’ to which we will add aggregations of all the relevant variables ijk as metadata. When any of the relevant variables abc changes, we will start writing a new ‘Batch’.

 

I see each Batch as either a CDF Event, or as a CDF Sequence or as a row in a Data Model. 

I expect the number of batches to grow into the millions as we have multiple production lines that are connected.


From the Cognite Docs, I understand that Events can be assigned to multiple Assets. I don’t forsee a need for this functionality in my use case. I understand that both Events and Sequences can store extra data (abc and values of averages of ijk etc.). It is written in the docs that Sequences are indexed on row number; I think this is worse for me as I think indexing by time would be most pertinent here. Thus I am leaning towards Events or a Data Model. 

Does anyone have experience of dealing with a similar use case? What would you use:

  • Events?
  • Sequences?
  • A specific data model?
  • A time series to mark changes in abc and on-the-fly calculations? 


Any ideas about differences between the approaches I described will be appreciated. 
 

3 replies

Userlevel 4
Badge

@APSHANKAR Sagar,
I suggest the raw data be stored as timeseries. (a, b, c, etc).  Each will have metadata describing the measurement.  You write each measurement value (i,j,k) in time (time:value) pair to the respective timeseries channel.
The concept of a batch is a sliding time processing window w(prev) and w(now).  You can write a Cognite function that runs on a schedule to compute the metric(s) you are after.  That metric can be output as a different timeseries.  You can then implement monitoring in Cognite Charts on that computed metric to alert on outliers.

Hope this helps,
Jason

Userlevel 1

Hello @Jason Dressel ,

abc, ijk are already time series. I think we need some data structures here as the metrics we are calculating pertain to not just timestamps but a time window and a combination of steady values of abc which define that time window. 

The question then is, if we are to use a data structure, which one (Events, Sequences FDM tables) is most suited. 

Userlevel 3

Hi Sagar,

From the APIs we have today, the best fit would probably be the Events API, however it lacks the aggregation capabilities you need across batches, once you’ve filtered by batch.

We are developing a new backend API, one which shares many of the common traits of DM, but which is optimised for high volume, log type data such as batch records.  The plan is to have this available in public beta toward the end of 2024 or early 2025.  Would you be interested in having a chat so I can ensure I understand your use case fully and for me to give you details on what we’re planning to build?

Kind Regards, Glen
 

Reply