Data workflow delay task

We recently encountered an issue where we had to delay a task in a workflow to wait for transformation metrics to become accurate. To solve this we put a sleep in a Cognite Function. We can also use external function calls to create a delay, but I was thinking that this might be a feature that would be nice to have built-in to Data Workflows.


Considering that there are times when you may want to delay a task for n minutes:

I was wondering if there are any plans to add a built-in delay task-type, or a delay field for the existing task-types, into Data Workflows?

e.g.

# maybe put it in the dependsOn field #############################
- externalId: someTask
  type: ...
  ...
  dependsOn:
    - externalId: transformationTaskId1
      delayMinutes: 10

# or as a dedicated field for each task ###########################
- externalId: someTask
  type: ...
  delayMinutes: 10
  dependsOn:
    ...

# or as a unique task type ########################################
- externalId: '10MinutesAfterTransforms'
  type: delay
  minutes: 10
  dependsOn:
    - transformationTaskId1
    - transformationTaskId2

- externalId: someTask
  type: ...
  dependsOn:
    - externalId: '10MinutesAfterTransforms'

And if not:
Is this something the Cognite team would consider adding?

Currently I think you can build a dedicated delay function by having a Cognite Function forward a delay instruction to an external endpoint using the isAsyncComplete flag, and have some external process tell the workflow to complete the task when the delay is over.

But I imagine this is the sort of thing that many teams who use Data Workflows would eventually need to build themselves.

You can also sleep inside the Cognite Function but, it’s wasteful since you’ll be paying for the time slept, it will increase the risk of the function timing out, and the function needs to “know” more about timing quirks of the other processes in the workflow.