Also, our function counting alarms uses multithreading for parallel processing. Running locally takes significantly shorter time than Cognite Function calls. Have anyone experienced runtime issues when using multithreading together with Cognite Functions?
Inside CDF Functions, you can click on the function details tab and see that each function only runs on one core and 1.5 GB RAM so I don’t think parallel processing will work on CDF Functions.
Hi @Vetle Nevland,
We are following up to see whether you're satisfied with the responses you've received?
@APSHANKAR Sagar Thanks for answering. Sorry for delayed response, just returned from vacation.
To clarify, we are counting and retrieving CDF Events concurrently using multiple threads with Python’s
ThreadPool
class (single or multiple cores), not parallel processing with multiple cores. Can Cognite Functions handle this?
Also, I still don’t understand why the Cognite Function executes for less than 2 minutes, while the backend code runs for more than 2 minutes. I see that multithreading is a known issue. Could this perhaps explain the observed deviations?
Hello @Vetle Nevland !
I work with the functions-team, so I can provide some insight here.
To answer your questions:
- Yes, using threads should be no problem.
- The “duration” measures the execution time of your `handle`-function in addition to any overhead incurred by our backend. This overhead includes the time it takes for various messages to propagate around our backend, as well as any scaling of compute resources.
Hope this helps! Let me know if you have any additional questions!
// Ivar Stangeby
