Skip to main content

I’m trying to upload a raster file to cdf, but I get this error message when trying to create the feature type. 

cognite.client.exceptions.CogniteAPIError: RASTER properties only support embedded storage. | code: 400 | X-Request-ID: 21781349-8ad8-9710-abdb-7c7199a268e7

raster_ft = FeatureType(
external_id="Raster_ft",
data_set_id=1,
properties={
"raster": {"type": "RASTER"}
}
)
client.geospatial.create_feature_types(raster_ft) # gives error message

Is this the right method to upload a raster to cdf? I found a method put_raster() in the docs, but it seems to require feature and feature type to be created already. The rest of my code: 
 

# create raster feature
feature = Feature(
external_id=f'raster.test',
data_set_id=1,
raster=open(raster_filepath) # What do I put here?
)
client.geospatial.create_features(feature_type_external_id='Raster_ft', feature=feature)

# looks like I need the feature with the raster property, do I need this function?
client.geospatial.put_raster(
feature_type_external_id=raster_ft.external_id,
feature_external_id=feature.external_id,
raster_property_name='raster',
raster_format='XYZ',
raster_srid='32633',
file=raster_filepath
)

 

It may not be documented, but the featuretype needs “storage”: “embedded”:

client.geospatial.create_feature_types(
FeatureType(
external_id="raster_ft",
properties={
"raster": {"srid": 4326, "type": "RASTER", "storage": "embedded", "optional": True},
"fileExtId": {"type": "STRING", "size": 256, "optional": True},
},
search_spec={"rasterIdx": {"properties": r"raster"]}},
)
)


Then you can ingest like this: 

 

def ingest_raster_files(ft, spec):
print(f"ingesting {spec} into {ft}")
features = e
Feature(external_id=itr"externalId"], file=it]"file"], tag=it""tag"])
for it in spec
]
client.geospatial.create_features(ft, features)
for f in features:
print(f)
client.geospatial.put_raster(
feature_type_external_id=ft,
feature_external_id=f.external_id,
raster_property_name="rast",
raster_format="GTiff",
raster_srid=4326,
file=f.file,
)


So it happens in two stages: create feature, then put raster to that feature.

Hope it helps!​​​​​​​


Hi Morten,

You need a feature type and define a raster property (together with other “conventional” properties as you see fit).

The raster property needs the “storage” parameter set to “embedded” and “optional to “True” so more like this:

raster_ft = FeatureType(
external_id="Raster_ft",
data_set_id=1,
properties={
"raster": {"type": "RASTER", "storage": "embedded", "optional": True}
}
)

Then create the feature that will receive the raster:

feature = Feature(
external_id=f'raster.test',
data_set_id=1,
)
client.geospatial.create_features(feature_type_external_id='Raster_ft', feature=feature)

And finally push the raster into the feature:

client.geospatial.put_raster(
feature_type_external_id=raster_ft.external_id,
feature_external_id=feature.external_id,
raster_property_name='raster',
raster_format='XYZ',
raster_srid='32633',
file=raster_filepath
)

If it helps, you can find some example code at the geospatial-examples

Vincent

 


Thank you for your replies! 

I tried again with these adjustments and a similar datasource as used in the example you linked Vincent (https://www.naturalearthdata.com/downloads/50m-raster-data/50m-natural-earth-2/). Now I run into a size limit of  50MB:

cognite.client.exceptions.CogniteAPIError: Embedded raster too large (52428800 bytes limit) | code: 413 | X-Request-ID: 9e403002-82f3-98bc-89f9-cac93c668a26

Some of our raster files are closer to 50GB, is CDF suited to store these files? 

 

When I try with our .tif files (less than 50MB) I get this error: 

cognite.client.exceptions.CogniteAPIError: Internal exception occurred. Please contact Geospatial Team with the following request id: 3daf7ab1-5de8-999a-9ae9-c80196551390. | code: 500 | X-Request-ID: 3daf7ab1-5de8-999a-9ae9-c80196551390

 


Hi Morten,

> Some of our raster files are closer to 50GB, is CDF suited to store these files? 

No, CDF only supports the embedded model. You have to split your large raster into smaller tiles in order to achieve that.

> X-Request-ID: 3daf7ab1-5de8-999a-9ae9-c80196551390

I see that the error has been logged. It seems to point to an invalid raster property. Can you double check the raster property name?

Vincent


Hi @Morten Movik,

Did the above help you?

 


Hi @Dilini Fernando

The answers were very helpful. We have decided not to store our geodata in cdf for now, considering the 50MB size-limit.


Reply