Skip to main content
Closed

Hosted Extractors Require Alerts when going offline

Related products:Extractors
  • September 6, 2024
  • 1 reply
  • 21 views

Forum|alt.badge.img+3

We have more than once now discovered our Time Series data which is currently being extracted using the Event Hub Hosted Extractor within the platform, has stopped extracting data for reasons we cannot determine.

 

To make hosted extractors production-worthy they must be able to notify platform owners when they fail, which they currently do not, and instead keep polling with bad session or connection info.

 

Here’s what this looks like in the logs provided by the hosted extractor Insight view:

For starters, this shouldn’t have 0 datapoints per hour.  That’s a potential opportunity for threshold alerting.

Expanding the details, these repeated failures are nondescript and do not indicate why things failed.  Discovering these issues reactively after platform users have asked why data is not trustable (things are missing), will present challenges for platform trust.

 

 

We are not yet into production with our platform and must find a solution to this before the data can be used by our teams.  

1 reply

Forum|alt.badge.img

These are all excellent suggestions we’re keeping in mind as we evolve and iterate on the Hosted Extractors feature!

 

And please accept this gentle reminder that from a product perspective, the Azure Event Hub hosted extractor is still in beta (not GA).  It’s currently scheduled for GA release together with the Q4 CDF release.