Solved

CogniteAPIError: RASTER properties only support embedded storage. | code: 400 |

17 days ago
12 April 2024
6 replies
44 views

Morten Movik
Active
2 replies

I’m trying to upload a raster file to cdf, but I get this error message when trying to create the feature type.

cognite.client.exceptions.CogniteAPIError: RASTER properties only support embedded storage. | code: 400 | X-Request-ID: 21781349-8ad8-9710-abdb-7c7199a268e7

raster_ft = FeatureType(
    external_id="Raster_ft",
    data_set_id=1,
    properties={
        "raster": {"type": "RASTER"}
    }
)
client.geospatial.create_feature_types(raster_ft)  # gives error message

Is this the right method to upload a raster to cdf? I found a method put_raster() in the docs, but it seems to require feature and feature type to be created already. The rest of my code:

# create raster feature
feature = Feature(
    external_id=f'raster.test',
    data_set_id=1,
    raster=open(raster_filepath)  # What do I put here?
)
client.geospatial.create_features(feature_type_external_id='Raster_ft', feature=feature)

# looks like I need the feature with the raster property, do I need this function?
client.geospatial.put_raster(
    feature_type_external_id=raster_ft.external_id,
    feature_external_id=feature.external_id,
    raster_property_name='raster',
    raster_format='XYZ',
    raster_srid='32633',
    file=raster_filepath
)

icon

Best answer by Vincent Gay 16 April 2024, 08:47

View original

6 replies

Userlevel 2

palronning
Architect
2 replies
17 days ago
12 April 2024

It may not be documented, but the featuretype needs “storage”: “embedded”:

client.geospatial.create_feature_types(
                FeatureType(
                    external_id="raster_ft",
                    properties={
                        "raster": {"srid": 4326, "type": "RASTER", "storage": "embedded", "optional": True},
                        "fileExtId": {"type": "STRING", "size": 256, "optional": True},
                    },
                    search_spec={"rasterIdx": {"properties": ["raster"]}},
                )
            )

Then you can ingest like this:

def ingest_raster_files(ft, spec):
    print(f"ingesting {spec} into {ft}")
    features = [
        Feature(external_id=it["externalId"], file=it["file"], tag=it["tag"])
        for it in spec
    ]
    client.geospatial.create_features(ft, features)
    for f in features:
        print(f)
        client.geospatial.put_raster(
            feature_type_external_id=ft,
            feature_external_id=f.external_id,
            raster_property_name="rast",
            raster_format="GTiff",
            raster_srid=4326,
            file=f.file,
        )

So it happens in two stages: create feature, then put raster to that feature.

Hope it helps!

Vincent Gay
Practitioner
4 replies
17 days ago
12 April 2024

Hi Morten,

You need a feature type and define a raster property (together with other “conventional” properties as you see fit).

The raster property needs the “storage” parameter set to “embedded” and “optional to “True” so more like this:

raster_ft = FeatureType(
    external_id="Raster_ft",
    data_set_id=1,
    properties={
        "raster": {"type": "RASTER", "storage": "embedded", "optional": True}
    }
)

Then create the feature that will receive the raster:

feature = Feature(
    external_id=f'raster.test',
    data_set_id=1,
)
client.geospatial.create_features(feature_type_external_id='Raster_ft', feature=feature)

And finally push the raster into the feature:

client.geospatial.put_raster(
    feature_type_external_id=raster_ft.external_id,
    feature_external_id=feature.external_id,
    raster_property_name='raster',
    raster_format='XYZ',
    raster_srid='32633',
    file=raster_filepath
)

If it helps, you can find some example code at the geospatial-examples

Vincent

Morten Movik
Author
Active
2 replies
14 days ago
15 April 2024

Thank you for your replies!

I tried again with these adjustments and a similar datasource as used in the example you linked Vincent (https://www.naturalearthdata.com/downloads/50m-raster-data/50m-natural-earth-2/). Now I run into a size limit of 50MB:

cognite.client.exceptions.CogniteAPIError: Embedded raster too large (52428800 bytes limit) | code: 413 | X-Request-ID: 9e403002-82f3-98bc-89f9-cac93c668a26

Some of our raster files are closer to 50GB, is CDF suited to store these files?

When I try with our .tif files (less than 50MB) I get this error:

cognite.client.exceptions.CogniteAPIError: Internal exception occurred. Please contact Geospatial Team with the following request id: 3daf7ab1-5de8-999a-9ae9-c80196551390. | code: 500 | X-Request-ID: 3daf7ab1-5de8-999a-9ae9-c80196551390