Hi There,
We have multiple CDF transformations running on schedule for every 15 minutes. Sometimes these jobs are taking more than 15 minutes and leading to Transformation failure with an error - ‘A job already runs for this transform ’. Is there any provision for us in CDF to not run a job/transformation while it is running already.
Hi :)
To my knowledge, there is no provision for that. If I am wrong, someone please correct me.
However, I would like to come with a suggestion on how to fix this.
My suggestion is to have 2 identical (except the name and id of course) transformations. Scheduled every 30 minutes but starting at different times, such that there is always at least one of them running every 15 minutes.
Note: This might lead to having both of them running a the same time (overlapping), so one needs to account for that. Or, before starting a transformation, add a step to check whether the other transformation is currently active. This can be done by querying a status table or a log where the start and end times of each transformation are recorded. But this requires to set up such table in place first.
Let me know what you think. :)
I will check the possibility of adding a audit table to know status of transformation.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
Check the
documentation
Ask the
Community
Take a look
at
Academy
Cognite
Status
Page
Contact
Cognite Support