We are looking into cloud-optimized storage formats such as GeoTIFF, Parquet, etc. One of the things we are trying to determine is whether we could fully utilize such cloud optimized formats with CDF Files.
Will the download-link returned by CDF Files allow us to do seek-operations and only download parts of the file?
Best answer by erlend.vollset
The Range header is part of HTTP, as specified by RFC7233 (https://datatracker.ietf.org/doc/html/rfc7233). Most blob stores will return an `Accept-Ranges` response header indicating support for Range requests. I’m not familiar with Pangeo, but if it conforms to this standard it should work fine.
Hi Thomas,
Thanks for reaching out!
The download endpoint supports supplying the Range header to specify which byte range to download, e.g.
Check the
documentation
Ask the
Community
Take a look
at
Academy
Cognite
Status
Page
Contact
Cognite Support
"Range": "bytes:1000-2000"Please not that both start and end are inclusive.
Does that solve your problem?
Allowing for a range-argument is a nice feature.
Is this part of a propriatary protocol, or can I (in python) use something like gcsfs as described here:
The Range header is part of HTTP, as specified by RFC7233 (https://datatracker.ietf.org/doc/html/rfc7233). Most blob stores will return an `Accept-Ranges` response header indicating support for Range requests. I’m not familiar with Pangeo, but if it conforms to this standard it should work fine.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.