Skip to main content

Hi!

I have a question regarding how quickly the metadata for a FunctionCall is updated after a successful call. The reason I ask is the following case:

I have deployed a set of Cognite Functions that run on a schedule every minute. In production this will be every 10 minutes, but for testing and debugging I want to speed it up to see how it works.

One of the functions perform this job:

1. Determine the time range for the data fetch based on response value of previous call (if first call we use pd.Timestamp.now())
2. Fetch Events in time range and process the data
3. Write processed events to a Sequence resource in CDF
4. Return a dict {"status": "success", "start_time": start_time}

The start_time of call N becomes the end_time for call N+1. The function is a “historic backfiller” that traverses back in time to a pre-defined date and fetches Events along the way.

The issue:

I have run into the scenario that in call N+1, when I get the response of call N, the status of call N is still Running. So I am wondering what the expected delays are for updating the FunctionCall’s attributes? 

It is not a critical issue in our case since we will run at a lower frequency, but if there are some limitations/expected delays here it would be great if it was mentioned in the docs. I expect it is a normal use case to use the response value of previous calls in the current call, which might be affected by such delays.

Hi @Anders Brakestad. Is it possible that Function call N is indeed still in progress when you start Function call N+1? Not sure how heavy the processing of event data within the Function really is, but if >1 minute it would not be finished before your next Function call starts. 

It might not be the right tool for your use case, but could also have a look at our new data workflows service, which will let you sequence multiple Function calls without using schedules to acheive it.


Hi @Jørgen Lund ,

Thanks for the reply. Our functions normally finish within 15 seconds, but sometimes the event fetch is larger which drags out the runtime.

I will read more about the workflows. This sounds like an interesting solution for deploying functionality to CDF. Could I leverage my existing functions if I deployed them without a schedule? Will the workflow deployment call the correct functions in the correct order? If so, then this sounds like a great simplification. I will read more about it.


Reply