Skip to main content
Oussama ALLALI
Seasoned ⭐️⭐️
Seasoned ⭐️⭐️
August 6, 2026
New

Feature request — native CDF Time Series push for metrics in PI Extractor and DB Extractor

Related products:Extractors
  • August 6, 2026
  • 3 replies
  • 30 views

Context

We currently run several custom Python extractors built on cognite-extractor-utils. For those, cognite-extractor-utils lets us configure a metrics.cognite block that periodically pushes the extractor's Prometheus
metrics (Counter/Gauge) directly to CDF Time Series — this gives us consistent,
queryable observability (row counts, run duration, throughput, failure counts, etc.)
across all our extractors, without any extra tooling.

Problem

The PI Extractor and the DB Extractor (official Cognite binaries) already expose
Prometheus metrics internally (e.g. pi_extractor_timeseries_states,
pi_extractor_stream_iterationspi_extractor_pi_data_pipe_events,
pi_extractor_streamer_data_pointspi_extractor_event_queue_size,
pi_extractor_pi_data_losspi_extractor_pi_connection_time,
pi_extractor_pi_connectionsextractor_utils_cdf_datapoints, …), but today they can
only be exposed via a local Prometheus scrape endpoint (metrics.server) or a Push
Gateway (metrics.push-gateways). There is no equivalent of metrics.cognite to push
these metrics natively to CDF Time Series, unlike our custom extractors.

Impact

Without native CDF Time Series push, we can't monitor these extractors the same way as our custom ones (no unified dashboarding/alerting in CDF/Grafana). We want to keep CDF as the single source of truth for observability, not add a separate tool just to collect metrics.

Request

Could you add support for a metrics.cognite configuration block (or equivalent) in the
PI Extractor and DB Extractor, consistent with the one already available via
cognite-extractor-utils, so their internal Prometheus metrics can be pushed
periodically to CDF Time Series (external-id prefix, target data set, push interval)?

3 replies

Mithila Jayalath
Expert ⭐️⭐️⭐️⭐️
Expert ⭐️⭐️⭐️⭐️
August 6, 2026

@Oussama ALLALI shall I convert this into a product idea?

Oussama ALLALI
Seasoned ⭐️⭐️
Seasoned ⭐️⭐️
August 6, 2026

yes please !

Mithila Jayalath
Expert ⭐️⭐️⭐️⭐️
Expert ⭐️⭐️⭐️⭐️
August 6, 2026

@Oussama ALLALI done. Please make sure to upvote the idea.