Check the
documentation
Ask the
Community
Take a look
at
Academy
Cognite
Status
Page
Contact
Cognite Support
Hello, I am trying to use “Filter time series” or “List time series” endpoints. However, because of 1000 limit, I need to parallelize these calls as I would like to retrieve thousands of timeseries. The challenge I am facing is the pagination used in these endpoints which is based on id of timeseries. Is there a way to use index as pagination?
Answer
Parallelize restAPI calls with pagination
Best answer by Jason Dressel
client: CogniteClient = ...
print(f"Max Workers: {client.config.max_workers}") #ensure you have workers >1 :)
timeseries = client.time_series.list(limit=None) #not making use of the workers
timeseries = client.time_series.list(partitions=10, limit=None) #will leverage the workers and run in parallel-Jason
Hi Diana, have you tried using the parallel retrieval method, using the /partition parameters?
Read here for more information
Hi Glen, thank you for getting back to me. Can you tell me more about the parallel retrieval method? It sounds like you may have shared a link, but I do not see it.
Hi Diane, sorry about that. Hopefully this link works!
https://developer.cognite.com/dev/concepts/pagination/
client: CogniteClient = ...
print(f"Max Workers: {client.config.max_workers}") #ensure you have workers >1 :)
timeseries = client.time_series.list(limit=None) #not making use of the workers
timeseries = client.time_series.list(partitions=10, limit=None) #will leverage the workers and run in parallel-Jason
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.