Skip to main content
Seasoned ⭐️⭐️
May 23, 2025
Solved

Newly encountered Read Timeout error

  • May 23, 2025
  • 11 replies
  • 157 views

Hi Hub,

 

We have some backend code running as a Github Action that fetches events using `clients.events.list` over a period of 1 year. The fetching is of course chunked into smaller time frames to avoid memory overload. This has worked just fine until recently where we are frequently encountering a

CogniteReadTimeout

error, which can be traced back to the following HTTP request error:

requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='az-ams-sp-002.cognitedata.com', port=443): Read timed out. (read timeout=30)

The error occurs inconsistently and not necessarily at the same point in time of the workflow run.

 

Have there been any recent changes/restrictions in the Cognite SDK that could explain the cause of the issue? As mentioned, some weeks ago the workflow succeeded without problems.

 

For additional context, we are using multiple threads to fetch events in parallel over metadata, for events within same time interval. Also, we use multiple cores to parallelize fetching and subsequent processing over different time intervals of the 1 year time frame.

 

Thanks!

 

Best regards,

Vetle.

Best answer by Sergei Voronichev

@Vetle Nevland Is it possible to slow down the writes with some controlled rate or reduce the number of parallel jobs?

11 replies

Seasoned ⭐️⭐️
June 2, 2025

@Sergei Voronichev Yes, have done some experimentation now.

 

Bad news:

  • We are only able to delete events from our own dataset, so unfortunately we can’t do much about the 340M events from other datasets.

Good news:

  • Increased read timeout to 90 seconds, and reduced number of events upserted in one go from 1000 to 100. Have done multiple deployment tests, and they now succeed :)

Conclusion:

  • I **assume** the root of the problem was rate limitations in combination with overload of CDF resources. We can only affect the former, but at least this seems to mitigate the error