Feature request — native CDF Time Series push for metrics in PI Extractor and DB Extractor
Context
We currently run several custom Python extractors built on cognite-extractor-utils. For those, cognite-extractor-utils lets us configure a metrics.cognite block that periodically pushes the extractor's Prometheus
metrics (Counter/Gauge) directly to CDF Time Series — this gives us consistent,
queryable observability (row counts, run duration, throughput, failure counts, etc.)
across all our extractors, without any extra tooling.
Problem
The PI Extractor and the DB Extractor (official Cognite binaries) already expose
Prometheus metrics internally (e.g. pi_extractor_timeseries_states,pi_extractor_stream_iterations, pi_extractor_pi_data_pipe_events,pi_extractor_streamer_data_points, pi_extractor_event_queue_size,pi_extractor_pi_data_loss, pi_extractor_pi_connection_time,pi_extractor_pi_connections, extractor_utils_cdf_datapoints, …), but today they can
only be exposed via a local Prometheus scrape endpoint (metrics.server) or a Push
Gateway (metrics.push-gateways). There is no equivalent of metrics.cognite to push
these metrics natively to CDF Time Series, unlike our custom extractors.
Impact
Without native CDF Time Series push, we can't monitor these extractors the same way as our custom ones (no unified dashboarding/alerting in CDF/Grafana). We want to keep CDF as the single source of truth for observability, not add a separate tool just to collect metrics.
Request
Could you add support for a metrics.cognite configuration block (or equivalent) in the
PI Extractor and DB Extractor, consistent with the one already available via
cognite-extractor-utils, so their internal Prometheus metrics can be pushed
periodically to CDF Time Series (external-id prefix, target data set, push interval)?
Check the
documentation
Ask the
Community
Take a look
at
Academy
Cognite
Status
Page
Contact
Cognite Support
