Skip to main content

Hi,

We have a large dataset containing more than 10,000 timeseries, and we are working on creating multiple subscriptions to accommodate them but again that can also be having 10k timeseries in sometime. Currently, we are using metadata with a specific property match as one of our filters, given that this metadata could also apply to 10,000 timeseries. we are considering adding a new filter based on createdTime. This would automatically include timeseries added after a specific time to a new subscription, avoiding the need to create new subscriptions manually.

Here is the filter we attempted to use:

"filter": {
  "and": :
    {
      "equals": {
        "property": p"dataSetId"],
        "value": "12346576588"
      }
    },
    {
      "range": {
        "property": b"createdTime"],
        "gte": 1719513000000
      }
    }
  ]
}
We encountered an internal server error (request ID: b45316a2-07ac-9ec8-986f-75bd66d698e4) .

Tried with creating createdTime as metadata as well, suspect there might be a limitation with the range filter on createdTime if we add in metadata filter.

Could you please advise on the correct way to add a createdTime filter in the subscription, or if you can suggest an alternative method to ensure new timeseries are automatically included in new subscriptions once the 10,000 limit is reached in one subscription?

Thank you for your report!
I can confirm that this functionality is missing in the backend, together with “all” and “any” filters. (“search” is purposefully exempt). I also believe the filters on createdTime/updatedTime may have been omitted completely.

Responding with 500 is clearly a bug. I will have to investigate whether there is a reason why the functionality is missing, in which case we will document the behavior, otherwise this is functionality we should implement.

Unfortunately, I can’t give a timeline for the next step, due to vacations, but we’ll get back to you

Is there any filter available or combination of filter that can be used to fetch new timeseries added to dataset after certain time(createdTime or lastUpdatedTime) ?

We have now added support for filtering on createdTime/lastUpdatedTime and rangeFilters. Please test it out and give feedback if something is not working as expected!

I believe the filter in the original post should work.

Note that createdTime and lastUpdatedTime are not necessarily related to when a time series is added to a dataset. The filter in the original post should match time series that were created in some form after the given createdTime and that belongs to the dataset right now.

“all” and “any” filters are still omitted, on purpose, as there are no relevant properties to apply them on.

@Rimmi Anand 

Hi @Rimmi Anand ,

After testing the filtering on createdTime/lastUpdatedTime and rangeFilters, please share any feedback or comments you may have. Your input is valuable to us!

Reply


Terms & ConditionsCookie Settings