We are having performance issues for event retrieval whenever there are more events even below 500.
Can you help me to understand how partitioning can be done or pagination, as we need to perform operation for each event after retrieval and send it as a api response.
I tried below:
for unstructre_insight in client.events(type="Insight", limit=None,partitions=10):
(“do something with unstructre_insight in each partition in parallel to reduce response time”)
I observed that if there are 64 events, then all 64 events are retrieved in one execution, how we can get one partition at a time and perform something for first partition in parallel while retrieving second partition to reduce time.
I also tried below:
Still it returned all 64 events
Getting same set of 10 insights, don't know how to get next set.
Hey Ankita, for me this way works fine
Could you please double-check that you are getting all 64 events at once?
I try to highlight several ways you can use the python SDK to retrieve your large number of events. The SDK auto supports paging. If you hit the API directly, you will need to navigate the cursors yourself.
Hope this helps,