Skip to main content
Solved

Filtering based on externalID


Hi,

To apply a startWith filter on a property, we can do:

prop_view = client.data_modeling.views.retrieve(('myspace', 'myview'))
filters.Prefix(prop_view[0].as_id().as_property_ref("myPropertyName"),"xxx")

This is working fine. However, as the externalId is not a property so the code below doesn’t work:

filters.Prefix(prop_view[0].as_id().as_property_ref("externalId"),"xxx")

and the error message is: 

“Invalid prefix filter with property reference [myspace, myview/94b12855a09cf4, externalId]. View myspace:myview/94b12855a09cf4 does not have property externalId.”

 

Could you help find out how to apply a startWith filter on externalId in python sdk?

 

Regards,

Xiaofeng

Best answer by Xiaofeng Wang

Thanks for the call @Jason Dressel . Now we got it to work using the dictionary style filter:

    filter_with_sdmId = {
        "and": [
            {
                "equals": {
                    "property": [
                        "node",
                        "space"
                    ],
                    "value": "xxx_Space"
                }
            },
            {
                "prefix": {
                    "property": [
                        "node",
                        "externalId"
                    ],
                    "value": "xxxID"
                }
            },
            {
                "equals": {
                    "property": [
                        "xxx_Space",
                        prop_view[0].as_id().as_source_identifier(),
                        "workflow",
                    ],
                    "value": "xxx"
                }
            }
        ]
    }

This syntax is not easy to find out through the API document as mentioned in the call. Hopefully it will be improved and add a bit more examples if the syntax is not very straightforward.

Furthermore, it would be good to get it work with PythonSDK which is more elegant. And the pythonSDK document should have more examples regarding how to do filtering.

Will flag this as resolved for now. Thanks for your help!

 

Regards,

Xiaofeng

View original
Did this topic help you find an answer to your question?

8 replies

Forum|alt.badge.img
  • Seasoned Practitioner
  • 281 replies
  • August 9, 2023

@Xiaofeng Wang,
ViewId type does not have a property externalId, but does have a property ‘external_id’ 😀

Are you trying to filter views based on external_id? If so, I’m a bit confused, because retrieving views by id explicitly will only retrieve one.


What type is the ‘filters’ object you have above?

-Jason


Hi Jason,

filters is a build in object in Cognite pythonSDK:

import cognite.client.data_classes.filters as filters

I am trying to filter instances, so the next step I did is:

filter_with_sdmId = filters.Prefix(prop_view[0].as_id().as_property_ref("workflow"),"xxx")

prop_timeseries_instances = client.data_modeling.instances.list(instance_type='node', sources=prop_view, limit=None, filter=filter_with_sdmId)

The code above works well, but I don’t know how to do filtering on externalId, instead of “workflow” property.

The external_id for a viewId is the name of the view, which doesn’t help in our case:

Our externalId of the instances looks like “XXXX:{instanceName}”, and I would like to find out all the instances with the “XXXX” as prefix in their externalId.

 

-Xiaofeng


Forum|alt.badge.img
  • Seasoned Practitioner
  • 281 replies
  • August 11, 2023

@Xiaofeng Wang,
Ok, so you’re filtering instances, not views.

Please find a working sample.  The filter is a dictionary which is the same as the JSON payload which you can find in the API docs.
 

from cognite.client import CogniteClient
from cognite.client.data_classes.data_modeling.spaces import Space, SpaceList
from cognite.client.data_classes.data_modeling import ViewList, View
from cognite.client.data_classes.data_modeling import ContainerList, Container
from cognite.client.data_classes.data_modeling import DataModelList, DataModel
from cognite.client.data_classes.data_modeling import NodeList, Node, EdgeList, Edge


client: CogniteClient = .... # Get your client your way

# The filter is a dictionary
emily_actors = {
        "and": [
            {
                "equals": {
                    "property": [
                        "node",
                        "space" 
                    ],
                    "value": "imdb"
                }
            },
            {
                "prefix": {
                    "property": [
                        "node",
                        "externalId"
                    ],
                    "value": "a-emily"
                }
            }
        ]
    }

nodes: NodeList = client.data_modeling.instances.list(instance_type='node', limit=None, filter=emily_actors)

Hope this helps!

-Jason


Thanks @Jason Dressel The filtering on the externalId works.

Could you also show me how to convert the code below to the dictionary format:

filters.Prefix(prop_view[0].as_id().as_property_ref("workflow"),"xxx")

Also it would be very helpful if you can let me know where I can find the how to compose the filters. Below is the example from the API document. and I could not make any meaning filter based on that sample. Is there any other documents with more detailed examples?

filter": {
"and": [
{
"in": {
"property": [
"tag"
],
"values": [
10011,
10011
]
}
},
{
"range": {
"property": [
"weight"
],
"gte": 0
}
}
]
}

Regards,

Xiaofeng


Forum|alt.badge.img
  • Seasoned Practitioner
  • 281 replies
  • August 16, 2023

@Xiaofeng Wang, Which project and which model are you wanting to query?  I can work to build examples from that.


-Jason


@Jason Dressel You can use xwsdm in slb_simuations for testing. We would like to query the property instances with Prefix filter on externalID(for example, ‘df023726-a8e9-43a2-8dfc-00578daf93a4’) and Equal filter on workflow(for example, ‘measured’)

 

Regards,

Xiaofeng


Forum|alt.badge.img
  • Seasoned Practitioner
  • 281 replies
  • August 17, 2023

@Xiaofeng Wang 
In the end, it was a bit easier to provide an imdb base example

This ‘and’ filter get’s all nodes with externalId prefixed with ‘a-emily’ AND in the space imdb.  

{
    "includeTyping": false,
    "instanceType": "node",
    "filter": {
        "and": [
            {
                "equals": {
                    "property": [
                        "node",
                        "space" // [space, externalid, createdTime, lastUpdatedTime]
                    ],
                    "value": "imdb"
                }
            },
            {
                "prefix": {
                    "property": [
                        "node",
                        "externalId"
                    ],
                    // Get me all emily actors
                    "value": "a-emily"
                }
            }
            // {
            //     "equals": {
            //         "property": [
            //             "node",
            //             "lastUpdatedTime" // [space, externalid, createdTime, lastUpdatedTime]
            //         ],
            //         "value": timestamp
            //     }
            // }
        ]
    }
}

This is an edge filter example that get’s edges for Movies.actors where the start node is a particular Movie node.

    "filter": {
        "and": [
            {
                "equals": {
                    "property": [
                        "edge",
                        "space" // [space, externalId, createdTime, lastUpdatedTime, startNode, endNode, type]
                    ],
                    "value": "imdb"
                }
            },
            {
                "equals": {
                    "property": [
                        "edge",
                        "type"
                    ],
                    "value": {
                        "space": "imdb",
                        "externalId": "Movie.actors"
                    }
                }
            },
            {
                "equals": {
                    "property": [
                        "edge",
                        "startNode"
                    ],
                    "value": {
                        "space": "imdb",
                        "externalId": "PK"
                    }
                }
            }
        ]
    }

I wasn’t able to deep link the specific location of the documentation, but I hope this screen shot helps you find further details:
 

If you get stuck, ping me and we can jump in a call.

-Jason


  • Author
  • Seasoned
  • 14 replies
  • Answer
  • August 17, 2023

Thanks for the call @Jason Dressel . Now we got it to work using the dictionary style filter:

    filter_with_sdmId = {
        "and": [
            {
                "equals": {
                    "property": [
                        "node",
                        "space"
                    ],
                    "value": "xxx_Space"
                }
            },
            {
                "prefix": {
                    "property": [
                        "node",
                        "externalId"
                    ],
                    "value": "xxxID"
                }
            },
            {
                "equals": {
                    "property": [
                        "xxx_Space",
                        prop_view[0].as_id().as_source_identifier(),
                        "workflow",
                    ],
                    "value": "xxx"
                }
            }
        ]
    }

This syntax is not easy to find out through the API document as mentioned in the call. Hopefully it will be improved and add a bit more examples if the syntax is not very straightforward.

Furthermore, it would be good to get it work with PythonSDK which is more elegant. And the pythonSDK document should have more examples regarding how to do filtering.

Will flag this as resolved for now. Thanks for your help!

 

Regards,

Xiaofeng


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings