Solved

Filtering based on externalID

  • 4 August 2023
  • 8 replies
  • 144 views
X
Rank

Hi,

To apply a startWith filter on a property, we can do:

prop_view = client.data_modeling.views.retrieve(('myspace', 'myview'))
filters.Prefix(prop_view[0].as_id().as_property_ref("myPropertyName"),"xxx")

This is working fine. However, as the externalId is not a property so the code below doesn’t work:

filters.Prefix(prop_view[0].as_id().as_property_ref("externalId"),"xxx")

and the error message is: 

“Invalid prefix filter with property reference [myspace, myview/94b12855a09cf4, externalId]. View myspace:myview/94b12855a09cf4 does not have property externalId.”

 

Could you help find out how to apply a startWith filter on externalId in python sdk?

 

Regards,

Xiaofeng

icon

Best answer by Xiaofeng Wang 17 August 2023, 21:53

View original

8 replies

J
Rank
Userlevel 4
Badge

@Xiaofeng Wang,
ViewId type does not have a property externalId, but does have a property ‘external_id’ 😀

Are you trying to filter views based on external_id? If so, I’m a bit confused, because retrieving views by id explicitly will only retrieve one.


What type is the ‘filters’ object you have above?

-Jason

X
Rank

Hi Jason,

filters is a build in object in Cognite pythonSDK:

import cognite.client.data_classes.filters as filters

I am trying to filter instances, so the next step I did is:

filter_with_sdmId = filters.Prefix(prop_view[0].as_id().as_property_ref("workflow"),"xxx")

prop_timeseries_instances = client.data_modeling.instances.list(instance_type='node', sources=prop_view, limit=None, filter=filter_with_sdmId)

The code above works well, but I don’t know how to do filtering on externalId, instead of “workflow” property.

The external_id for a viewId is the name of the view, which doesn’t help in our case:

Our externalId of the instances looks like “XXXX:{instanceName}”, and I would like to find out all the instances with the “XXXX” as prefix in their externalId.

 

-Xiaofeng

J
Rank
Userlevel 4
Badge

@Xiaofeng Wang,
Ok, so you’re filtering instances, not views.

Please find a working sample.  The filter is a dictionary which is the same as the JSON payload which you can find in the API docs.
 

from cognite.client import CogniteClient
from cognite.client.data_classes.data_modeling.spaces import Space, SpaceList
from cognite.client.data_classes.data_modeling import ViewList, View
from cognite.client.data_classes.data_modeling import ContainerList, Container
from cognite.client.data_classes.data_modeling import DataModelList, DataModel
from cognite.client.data_classes.data_modeling import NodeList, Node, EdgeList, Edge


client: CogniteClient = .... # Get your client your way

# The filter is a dictionary
emily_actors = {
        "and": [
            {
                "equals": {
                    "property": [
                        "node",
                        "space" 
                    ],
                    "value": "imdb"
                }
            },
            {
                "prefix": {
                    "property": [
                        "node",
                        "externalId"
                    ],
                    "value": "a-emily"
                }
            }
        ]
    }

nodes: NodeList = client.data_modeling.instances.list(instance_type='node', limit=None, filter=emily_actors)

Hope this helps!

-Jason

X
Rank

Thanks @Jason Dressel The filtering on the externalId works.

Could you also show me how to convert the code below to the dictionary format:

filters.Prefix(prop_view[0].as_id().as_property_ref("workflow"),"xxx")

Also it would be very helpful if you can let me know where I can find the how to compose the filters. Below is the example from the API document. and I could not make any meaning filter based on that sample. Is there any other documents with more detailed examples?

filter": {
"and": [
{
"in": {
"property": [
"tag"
],
"values": [
10011,
10011
]
}
},
{
"range": {
"property": [
"weight"
],
"gte": 0
}
}
]
}

Regards,

Xiaofeng

J
Rank
Userlevel 4
Badge

@Xiaofeng Wang, Which project and which model are you wanting to query?  I can work to build examples from that.


-Jason

X
Rank

@Jason Dressel You can use xwsdm in slb_simuations for testing. We would like to query the property instances with Prefix filter on externalID(for example, ‘df023726-a8e9-43a2-8dfc-00578daf93a4’) and Equal filter on workflow(for example, ‘measured’)

 

Regards,

Xiaofeng

J
Rank
Userlevel 4
Badge

@Xiaofeng Wang 
In the end, it was a bit easier to provide an imdb base example

This ‘and’ filter get’s all nodes with externalId prefixed with ‘a-emily’ AND in the space imdb.  

{
    "includeTyping": false,
    "instanceType": "node",
    "filter": {
        "and": [
            {
                "equals": {
                    "property": [
                        "node",
                        "space" // [space, externalid, createdTime, lastUpdatedTime]
                    ],
                    "value": "imdb"
                }
            },
            {
                "prefix": {
                    "property": [
                        "node",
                        "externalId"
                    ],
                    // Get me all emily actors
                    "value": "a-emily"
                }
            }
            // {
            //     "equals": {
            //         "property": [
            //             "node",
            //             "lastUpdatedTime" // [space, externalid, createdTime, lastUpdatedTime]
            //         ],
            //         "value": timestamp
            //     }
            // }
        ]
    }
}

This is an edge filter example that get’s edges for Movies.actors where the start node is a particular Movie node.

    "filter": {
        "and": [
            {
                "equals": {
                    "property": [
                        "edge",
                        "space" // [space, externalId, createdTime, lastUpdatedTime, startNode, endNode, type]
                    ],
                    "value": "imdb"
                }
            },
            {
                "equals": {
                    "property": [
                        "edge",
                        "type"
                    ],
                    "value": {
                        "space": "imdb",
                        "externalId": "Movie.actors"
                    }
                }
            },
            {
                "equals": {
                    "property": [
                        "edge",
                        "startNode"
                    ],
                    "value": {
                        "space": "imdb",
                        "externalId": "PK"
                    }
                }
            }
        ]
    }

I wasn’t able to deep link the specific location of the documentation, but I hope this screen shot helps you find further details:
 

If you get stuck, ping me and we can jump in a call.

-Jason

X
Rank

Thanks for the call @Jason Dressel . Now we got it to work using the dictionary style filter:

    filter_with_sdmId = {
        "and": [
            {
                "equals": {
                    "property": [
                        "node",
                        "space"
                    ],
                    "value": "xxx_Space"
                }
            },
            {
                "prefix": {
                    "property": [
                        "node",
                        "externalId"
                    ],
                    "value": "xxxID"
                }
            },
            {
                "equals": {
                    "property": [
                        "xxx_Space",
                        prop_view[0].as_id().as_source_identifier(),
                        "workflow",
                    ],
                    "value": "xxx"
                }
            }
        ]
    }

This syntax is not easy to find out through the API document as mentioned in the call. Hopefully it will be improved and add a bit more examples if the syntax is not very straightforward.

Furthermore, it would be good to get it work with PythonSDK which is more elegant. And the pythonSDK document should have more examples regarding how to do filtering.

Will flag this as resolved for now. Thanks for your help!

 

Regards,

Xiaofeng

Reply


Terms & ConditionsCookie Settings