Cognite Data Fusion: Support Dates Prior to 1970 in API and Python SDK

Related products: API and SDKs

We have a time series that begins on the install date of an asset, which for some assets out in the field have a install date prior to 1970.  When we try to insert these time series data points into CDF via the Python SDK, we are getting this error:

 cognite.client.exceptions.CogniteAPIError: Timestamp is too low, must be at least 31536000000

 

Is there already a workaround for storing data points with a date prior to 1970? Is this supported by the HTTP API, but not the Python SDK?

 

Thank you,

Ben Brandt

Updated idea statusNewGathering Interest

Hi Ben,

This is actually an API limitation that the Python SDK steps into.

We do not currently support timestamps prior to Jan 1st, 1971 for a time series (so “no” to your 2nd question).

The underlying codebase can support timestamps from 1900 to 2150, but changing the API is being evaluated (impact to the API, existing solutions, etc).

I’m creating a ticket for the request so our engineers can start thinking more structured about it, and describe possible workarounds, etc. 

Thank you for the report!

 


Thank you for the response, Thomas.

 

I have heard from our product owner, @Mark Felder, that dates after 2050 will be needed for us for storing predictive results, but future is not as urgent as historical.  Thanks!


Thanks Thomas.

It seems logical that the API would be able to support the same range of timestamps that the underlying codebase supports.

Looking forward to hearing more on a potential solution.


It seems logical that the API would be able to support the same range of timestamps that the underlying codebase supports.

 

Agreed @Mark Felder. We do need to make sure we’re not breaking anything when we allow dates based on the fully refactored, multi-cloud support capable software stack that is now underlying the Time Series API. The constraints w.r.t to dates prior to 1971 are from the old stack (which we migrated off of before the holidays). 

So we’re having to plan some work in order to implement support for dates prior to 1971 and up to (potentially) Dec 31. 2150.


Gathering InterestPlanned for development

Looks like there’s been progress.  Docs now say this:

Timestamps are in the range from 00:00, January 1st, 1900 to 23:59, December 31st, 2050. Negative timestamps are used to define dates before 1970

 

I was able to run a Synthetic Time Series Query in Postman with a negative “start”.  Now it’s up to the individual SDK’s to update their validations for start.

 

Any plans to move end out further into the future?  The max 64 bit integer, 9,223,372,036,854,775,807, can get us pretty far out there.


Confirmed: Minimum TS now seems to be -2208988800000.

Date and time (GMT): Monday, January 1, 1900 12:00:00 AM

 

{

    "error": {

        "code": 400,

        "message": "Timestamp cannot be lower than -2208988800000. Was '-2211753600000'"

    }

}


Planned for developmentImplemented