Hi @Dilini Fernando,
The answers were very helpful. We have decided not to store our geodata in cdf for now, considering the 50MB size-limit.
Hi Morten,
> Some of our raster files are closer to 50GB, is CDF suited to store these files?
No, CDF only supports the embedded model. You have to split your large raster into smaller tiles in order to achieve that.
> X-Request-ID: 3daf7ab1-5de8-999a-9ae9-c80196551390
I see that the error has been logged. It seems to point to an invalid raster property. Can you double check the raster property name?
Vincent
Thank you for your replies!
I tried again with these adjustments and a similar datasource as used in the example you linked Vincent (https://www.naturalearthdata.com/downloads/50m-raster-data/50m-natural-earth-2/). Now I run into a size limit of 50MB:
cognite.client.exceptions.CogniteAPIError: Embedded raster too large (52428800 bytes limit) | code: 413 | X-Request-ID: 9e403002-82f3-98bc-89f9-cac93c668a26
Some of our raster files are closer to 50GB, is CDF suited to store these files?
When I try with our .tif files (less than 50MB) I get this error:
cognite.client.exceptions.CogniteAPIError: Internal exception occurred. Please contact Geospatial Team with the following request id: 3daf7ab1-5de8-999a-9ae9-c80196551390. | code: 500 | X-Request-ID: 3daf7ab1-5de8-999a-9ae9-c80196551390
Hi Morten,
You need a feature type and define a raster property (together with other “conventional” properties as you see fit).
The raster property needs the “storage” parameter set to “embedded” and “optional to “True” so more like this:
raster_ft = FeatureType(
external_id="Raster_ft",
data_set_id=1,
properties={
"raster": {"type": "RASTER", "storage": "embedded", "optional": True}
}
)
Then create the feature that will receive the raster:
feature = Feature(
external_id=f'raster.test',
data_set_id=1,
)
client.geospatial.create_features(feature_type_external_id='Raster_ft', feature=feature)
And finally push the raster into the feature:
client.geospatial.put_raster(
feature_type_external_id=raster_ft.external_id,
feature_external_id=feature.external_id,
raster_property_name='raster',
raster_format='XYZ',
raster_srid='32633',
file=raster_filepath
)
If it helps, you can find some example code at the geospatial-examples
Vincent
It may not be documented, but the featuretype needs “storage”: “embedded”:
client.geospatial.create_feature_types(
FeatureType(
external_id="raster_ft",
properties={
"raster": {"srid": 4326, "type": "RASTER", "storage": "embedded", "optional": True},
"fileExtId": {"type": "STRING", "size": 256, "optional": True},
},
search_spec={"rasterIdx": {"properties": r"raster"]}},
)
)
Then you can ingest like this:
def ingest_raster_files(ft, spec):
print(f"ingesting {spec} into {ft}")
features = e
Feature(external_id=itr"externalId"], file=it]"file"], tag=it""tag"])
for it in spec
]
client.geospatial.create_features(ft, features)
for f in features:
print(f)
client.geospatial.put_raster(
feature_type_external_id=ft,
feature_external_id=f.external_id,
raster_property_name="rast",
raster_format="GTiff",
raster_srid=4326,
file=f.file,
)
So it happens in two stages: create feature, then put raster to that feature.
Hope it helps!