Skip to main content

How to access reverse direct relations through sdk


Forum|alt.badge.img

Hello team, 

I am trying to get the details of a reverse direct relation from my data model. In the SDK documentation I found some sdk methods. I have tried client.data_modeling.views.ReverseDirectRelation(source= "Entity", through="entityType") but I am getting the error that ReverseDirectRelation is not found in views could you please tell me what SDK method I should use to retrieve information on ReverseDirectRelations from my models.

19 replies

Anders  Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 96 replies
  • October 16, 2024

Hi @Gargi Bhosale ,

You cannot get the reverse direct relations using the .retrieve or .list methods, you have to use the .query method. Typically, you do have to use the `NodeResultExpression` with `node.through` set to the direct relation property of the view on the other side of the reverse direct relations and the `nodes.direction=’inwards’`. 

An alternative is to use `pygen`. `pygen` generates a tailor made SDK for your data model. In the generated, `.list` method there is a parameter `retrieve_connections` that you can use to retrieve connections. This will do the `.list` or `.query` call for you. See https://cognite-pygen.readthedocs-hosted.com/en/latest/usage/listing_filtering_retrieving.html#reading-instances-list-and-retrieve


Forum|alt.badge.img
  • Author
  • Committed
  • 20 replies
  • October 17, 2024

I see that pygen is highly specific on a particular model. In my case ,I intend to make the code generic to any model the user would select. In that case, would you recommend using the .query method? Are there any performance differences between the both?


Anders  Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 96 replies
  • October 21, 2024

There are two cases.

  1. If the direct relation of the other side is a single direct relation, use the query method. client.data_modeling.instances.query(...) You can see an example in the pygen example here.
  2. If the direct relation of the other side is a list of direct relations, use the search method.
    client.data_modeling.instances.search(...). You will do a lookup on the nodes with the direct relation filtering for matching the nodes with the reverse direct relation.
     

Forum|alt.badge.img
  • Author
  • Committed
  • 20 replies
  • October 23, 2024

The pygen example for client.data_modeling.instances.query(...) is not very clear and what I am trying is not working.

this is what I am trying can you please correct?

 

from cognite.client.data_classes.data_modeling.query import Query, Select, NodeResultSetExpression, EdgeResultSetExpression, SourceSelector
view=ViewId(space='slb-pdm-dm-governed', external_id='Property', version='1_7')
nested_prop_view_id = ViewId(space='slb-pdm-dm-governed', external_id='PropertyType', version='1_6')
properties = ["externalId", "propertyType"]
nested_prop_view_properties=["externalId"]

query= Query(
    with_={"PropertyType":NodeResultSetExpression(
                        from_="Property",
                        direction="inwards",
                        filter= HasData(views=[nested_prop_view_id]),
                        through=nested_prop_view_id.as_property_ref("directSingle"),
                    )},
    select={"PropertyType": Select([SourceSelector(nested_prop_view_id, nested_prop_view_properties)])}
)
res1 = client.data_modeling.instances.query(query)

 


Forum|alt.badge.img
  • Author
  • Committed
  • 20 replies
  • October 24, 2024

Another important observation: 

when I use client.data_modeling.instances.search(...) without the query because I want everything and with limit= None, I get only 100 records, whereas I have much more than them. When I increase the limit to 1000 it gives 1000 records, I have more than 1000 as well. Ideally, I expect all the data to come when I pass limit=None.

 


Anders  Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 96 replies
  • October 28, 2024

Give me a few days, and I will create a full example in the SDK docs on how to do this.


Forum|alt.badge.img
  • Author
  • Committed
  • 20 replies
  • November 6, 2024

Sure Anders, let me know when it is available


Anders  Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 96 replies
  • November 9, 2024

Forum|alt.badge.img
  • Author
  • Committed
  • 20 replies
  • November 11, 2024

Thank you, so when I use the search() for reverse direct relation of listable relation, I see that we get 1000 instances by using client.data_modeling.instances.search(activity_view, filter=is_equipment, limit=1_000)

I am not getting any cursor or any other way to retrieve the full data, all the instances satisfied with the query, can you please help me with this?


@Anders Albert  - We still are not able to get the complete data with the temporarily approach or query suggested ? Do you see any code change is required in sdk or some other way of querying is required.

We are blocked as of now due to this.

Appreciate if you can expedite your response here.

Regards,
Neeraj B


Anders  Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 96 replies
  • November 17, 2024

@Gargi Bhosale Search do not support pagination. I have extended the example to show how you can do a lookup on each individual equipment instead to ensure you get all activities: https://cognite-pygen.readthedocs-hosted.com/en/latest/docs-dev/docs_query.html#limitation-with-pagination


Forum|alt.badge.img
  • Author
  • Committed
  • 20 replies
  • November 18, 2024

Hello ​@Anders Albert , 

I cannot make sense what is written in the link. I assume there is a problem in formatting, could you please give the example in a better format so that I can understand? :)

 


Anders  Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 96 replies
  • November 19, 2024

Seems the documentation was broken. Should be fixed now. Can you try again?

 

https://cognite-pygen.readthedocs-hosted.com/en/latest/docs-dev/docs_query.html#limitation-with-pagination


Forum|alt.badge.img
  • Author
  • Committed
  • 20 replies
  • November 27, 2024

Hello Anders,

 

I read the documentation. It seems that we are assuming in the example that one equipment can have maximum 1000 activities only. Two observations on this from our side:

  1. In our use case we can have more than 1000 instances for one equipment.
  2. Filtering approach will not be performant as we process a lot of data in our application.

We want to understand:

  1. Any strong reason for not supporting this in query SDK.
  2. If the above feature is not possible for some reason, for a performant solution we would need support for pagination in search.

 

 


Anders  Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 96 replies
  • November 27, 2024

This is a limitation on the server side with the /search endpoint. If you want to do an exhaustive, you have the use the /list endpoint, which is implemented in the `.list(...)` method in the SDK.

This is a trade-off. Search is high-performant good for fetching specific items fast. The `.list()` is slower but supports pagination (baked in to the SDK) so you can do an exhaustive listing.  

 


Haaland
Committed
Forum|alt.badge.img
  • Committed
  • 16 replies
  • December 19, 2024

Hi ​@Anders Albert ,

 

Say you have a reverse direct relation: 

Equipment <---- Activities

That is,

Activities has a relation to Equipment  and other properties defined as

equipment:[Equipment]
property1: String

and

Equipment as a reverse direct relation to Activities defined as

activity: [Activities]
@reverseDirectRelation(throughProperty: "equipment")

 

How would you list all instances in Equipment filtered on properties in Activities?

 

I tried something like (i.e. tried to traverse the relation) but this times out :

client.data_modeling.instances.list(
        instance_type="node",
        sources=EQUIPMENT_VIEW_ID,
        filter=Nested(
                 scope=EQUIPMENT_VIEW_ID.as_property_ref("activity"), 
                 filter=In(ACTIVITY_VIEW_ID.as_property_ref('property1'), ["x", "y"])
               ),
        limit=None
        )

This works if the relation is a normal relation (i.e. going the other way from Activites to Equipment), but not from Equipment to Activities.

 

Secondly, because of this seeming limitation, are there benefits for having reverse direct relations?


Anders  Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 96 replies
  • December 19, 2024

You do the lookup on the activity and not the equipment. 

Something like this

client.data_modeling.instances.list( instance_type="node", sources=ACTIVITY_VIEW_ID, filter=In(ACTIVITY_VIEW_ID.as_property_ref('property1'), ["x", "y"]), limit=None )

 


Haaland
Committed
Forum|alt.badge.img
  • Committed
  • 16 replies
  • December 20, 2024

Hi ​@Anders Albert ,

Apologies, I see now my question set up was incomplete.

Say, Equipment and Activities are defined as below: 

type Equipment
  @view(version: "1")
{
  property1: String
  activity: [Activities]
    @reverseDirectRelation(throughProperty: "equipment")
}


type Activities
  @view(version: "1")
{
  property2: String
  equipment: Equipment
}

What’s the best way to retrieve Equipment instances filtered on property1 (Equipment property) and property2 (Activities property)?

Something like this times out:

client.data_modeling.instances.list(
        instance_type="node",
        sources=EQUIPMENT_VIEW_ID,
        filter=And(
                 In(EQUIPMENT_VIEW_ID.as_property_ref("property1"), ["a", "b"],
                 Nested(
                 scope=EQUIPMENT_VIEW_ID.as_property_ref("activity"), 
                 filter=In(ACTIVITY_VIEW_ID.as_property_ref('property2'), ["x", "y"])
                 )
               ),
        limit=None
        )

 

 


Anders  Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 96 replies
  • December 30, 2024

I wrote up an article to try to address your issue. I think the issue is likely lack of indices on the properties you are filtering on, causing the timeout. Workaround is to try to split the query into two

I published it temporarily here: https://cognite-pygen.readthedocs-hosted.com/en/latest/docs-dev/query_filter.html


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings