Skip to main content

How to: Transformation Execution Timestamps Explained [Cognite Official]

  • September 1, 2025
  • 1 reply
  • 31 views
hasitha malinga
Practitioner
Forum|alt.badge.img

When running Transformations, multiple timestamps are recorded to track the job lifecycle. Users often see differences between these times, which can lead to confusion. This document clarifies what each field means and how duration is calculated.

 

1. Created Time

  • Definition: The time when the user (or system) requested the transformation to run.
  • User expectation vs. reality: Many assume this is when the job begins execution. However, there may be a delay between job submission and actual start.
  • Possible causes of delay:
    • Resource availability in the cluster.
    • Internal scheduling/queueing.

2. Started Time

  • Definition: The time when the transformation actually begins execution (Spark driver starts running the job).
  • Key difference from Created: There may be seconds or even minutes of gap between submission and execution start, depending on system load.
  • UI representation: Sometimes shown simply as 'STARTED,' which has caused confusion. A clearer label like 'Run Started' is being considered.

3. Finished Time

  • Definition: The time when the transformation finished execution, either successfully or with failure.
  • Note: This represents the end of processing, not the time results are ingested.

4. Duration

  • Currently calculated as:
    • Duration = Started Time to  Finished Time
  • What it represents: The actual processing time of the transformation job.
  • Why not include Created → Started delay?
    • If duration included the waiting/queueing time, users might confuse cluster delays with actual job performance.
    • Current calculation reflects the true runtime of the transformation.

5. UI/Label Improvements (in progress)

To make these labels clearer in the Transformation UI, the following improvements are proposed:

  • Relabeling fields for better clarity:
    • 'Created' → Job Submitted
    • 'Started' → Run Started
    • 'Finished' → Run Finished
Dilini Fernando
Mithila Jayalath
Shashan Udawatte

1 reply

Peter Arwanitis
Practitioner
Forum|alt.badge.img

Thank you for the write-up!

There is on more time, that is required to check the run-time during a transformation-run, which is last_seen_time.

 

In Python a code snippet to calculate duration looks like:

# tsj => time-series job, you can get programmatic with "client.transformations.jobs.list(..)" which returns an object of type "TransformationJobList"

import datetime

duration = datetime.timedelta(milliseconds=(tsj.finished_time or tsj.last_seen_time or tsj.created_time) - tsj.started_time)

 

Terms & ConditionsCookie SettingsAccessibility statement