Skip to main content
Implemented

Cognite Data Fusion: Support Dates Prior to 1970 in API and Python SDK

Related products:REST API
  • April 11, 2022
  • 9 replies
  • 120 views

Ben Brandt
Seasoned ⭐️⭐️⭐️

We have a time series that begins on the install date of an asset, which for some assets out in the field have a install date prior to 1970.  When we try to insert these time series data points into CDF via the Python SDK, we are getting this error:

 cognite.client.exceptions.CogniteAPIError: Timestamp is too low, must be at least 31536000000

 

Is there already a workaround for storing data points with a date prior to 1970? Is this supported by the HTTP API, but not the Python SDK?

 

Thank you,

Ben Brandt

9 replies

Anita Hæhre
Expert ⭐️⭐️⭐️⭐️
Forum|alt.badge.img+3
  • Head of Community
  • April 12, 2022
Updated idea statusNewGathering Interest

Forum|alt.badge.img

Hi Ben,

This is actually an API limitation that the Python SDK steps into.

We do not currently support timestamps prior to Jan 1st, 1971 for a time series (so “no” to your 2nd question).

The underlying codebase can support timestamps from 1900 to 2150, but changing the API is being evaluated (impact to the API, existing solutions, etc).

I’m creating a ticket for the request so our engineers can start thinking more structured about it, and describe possible workarounds, etc. 

Thank you for the report!

 


Ben Brandt
Seasoned ⭐️⭐️⭐️
  • Author
  • Seasoned ⭐️⭐️⭐️
  • April 13, 2022

Thank you for the response, Thomas.

 

I have heard from our product owner, @Mark Felder, that dates after 2050 will be needed for us for storing predictive results, but future is not as urgent as historical.  Thanks!


Forum|alt.badge.img
  • Active ⭐️⭐️⭐️
  • April 13, 2022

Thanks Thomas.

It seems logical that the API would be able to support the same range of timestamps that the underlying codebase supports.

Looking forward to hearing more on a potential solution.


Forum|alt.badge.img

It seems logical that the API would be able to support the same range of timestamps that the underlying codebase supports.

 

Agreed @Mark Felder. We do need to make sure we’re not breaking anything when we allow dates based on the fully refactored, multi-cloud support capable software stack that is now underlying the Time Series API. The constraints w.r.t to dates prior to 1971 are from the old stack (which we migrated off of before the holidays). 

So we’re having to plan some work in order to implement support for dates prior to 1971 and up to (potentially) Dec 31. 2150.


Anita Hæhre
Expert ⭐️⭐️⭐️⭐️
Forum|alt.badge.img+3
  • Head of Community
  • August 9, 2022
Gathering InterestPlanned for development

Ben Brandt
Seasoned ⭐️⭐️⭐️
  • Author
  • Seasoned ⭐️⭐️⭐️
  • August 17, 2022

Looks like there’s been progress.  Docs now say this:

Timestamps are in the range from 00:00, January 1st, 1900 to 23:59, December 31st, 2050. Negative timestamps are used to define dates before 1970

 

I was able to run a Synthetic Time Series Query in Postman with a negative “start”.  Now it’s up to the individual SDK’s to update their validations for start.

 

Any plans to move end out further into the future?  The max 64 bit integer, 9,223,372,036,854,775,807, can get us pretty far out there.


Ben Brandt
Seasoned ⭐️⭐️⭐️
  • Author
  • Seasoned ⭐️⭐️⭐️
  • August 17, 2022

Confirmed: Minimum TS now seems to be -2208988800000.

Date and time (GMT): Monday, January 1, 1900 12:00:00 AM

 

{

    "error": {

        "code": 400,

        "message": "Timestamp cannot be lower than -2208988800000. Was '-2211753600000'"

    }

}


Anita Hæhre
Expert ⭐️⭐️⭐️⭐️
Forum|alt.badge.img+3
  • Head of Community
  • October 27, 2022
Planned for developmentImplemented