Skip to main content
Solved

Problem with ingesting datapoint immediately after Timeseries object creation with multiple concurrent jobs.


We’re attempting to execute 10 concurrent jobs, and each one will create approximately 20 Timeseries objects, immediately ingesting datapoints for these newly created Timeseries objects in a loop.

However, if we try to run 500 similar jobs with 10 concurrently running at a time, about 10% of the jobs fail due to a ‘Timeseries not found’ error.

We theorize that the creation of a Timeseries object can take some time. Therefore, immediately ingesting the datapoint might fail because Cognite doesn’t refresh quickly enough to find the corresponding Timeseries object. This issue only occurs when we attempt to run multiple concurrent jobs.

Has anyone else encountered similar issues?

 

Best answer by Dilini Fernando

Hi @Dexter Nguyen,

I hope the above helped. I’m closing this topic now, please feel free to create a new post if you still experience problems. 

Br,

Dilini

View original
Did this topic help you find an answer to your question?

Forum|alt.badge.img

@Dexter Nguyen 

Is there an potential race condition here?  Would a thread try to upload datapoints before the timeseries is actually created?  Given that you only observe this behavior during concurrent jobs, this seems likely.  Is is possible to have 2 jobs types: First, create all the necessary timeseries objects. You can POST to create 1000 time series objects in a single request.  Second, post to create the datapoints?

-Jason


Glen Sykes
Seasoned Practitioner
  • Seasoned Practitioner
  • November 3, 2023

Thanks Jason, I was just about to add:

Did you verify that you got a 201 response upon creation of the Time Series before attempting to ingest the data?  It could be that you’re attempting to write the data to the Time Series before it’s fully created, or if creation failed for some reason.

We do have integration tests doing this operation all of the time, running concurrently without issue.


Dilini Fernando
Seasoned Practitioner
Forum|alt.badge.img+2

Hi @Dexter Nguyen,

We are following up to see whether you're satisfied with the responses you've received.

Best regards,
Dilini 


Dilini Fernando
Seasoned Practitioner
Forum|alt.badge.img+2

Hi @Dexter Nguyen,

I hope the above helped. I’m closing this topic now, please feel free to create a new post if you still experience problems. 

Br,

Dilini


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings