How to access reverse direct relations through sdk

​@Gargi Bhosale Search do not support pagination. I have extended the example to show how you can do a lookup on each individual equipment instead to ensure you get all activities: https://cognite-pygen.readthedocs-hosted.com/en/latest/docs-dev/docs_query.html#limitation-with-pagination

Hello ​@Anders Albert , 

I cannot make sense what is written in the link. I assume there is a problem in formatting, could you please give the example in a better format so that I can understand? :)

Seems the documentation was broken. Should be fixed now. Can you try again?

https://cognite-pygen.readthedocs-hosted.com/en/latest/docs-dev/docs_query.html#limitation-with-pagination

Hello Anders,

I read the documentation. It seems that we are assuming in the example that one equipment can have maximum 1000 activities only. Two observations on this from our side:

1. In our use case we can have more than 1000 instances for one equipment.
2. Filtering approach will not be performant as we process a lot of data in our application.

We want to understand:

1. Any strong reason for not supporting this in query SDK.
2. If the above feature is not possible for some reason, for a performant solution we would need support for pagination in search.

This is a limitation on the server side with the /search endpoint. If you want to do an exhaustive, you have the use the /list endpoint, which is implemented in the `.list(...)` method in the SDK.

This is a trade-off. Search is high-performant good for fetching specific items fast. The `.list()` is slower but supports pagination (baked in to the SDK) so you can do an exhaustive listing.  

Hi ​@Anders Albert ,

Say you have a reverse direct relation: 

Equipment <---- Activities

That is,

Activities has a relation to Equipment  and other properties defined as

equipment:[Equipment]
property1: String

and

Equipment as a reverse direct relation to Activities defined as

activity: [Activities]
@reverseDirectRelation(throughProperty: "equipment")

How would you list all instances in Equipment filtered on properties in Activities?

I tried something like (i.e. tried to traverse the relation) but this times out :

client.data_modeling.instances.list(
        instance_type="node",
        sources=EQUIPMENT_VIEW_ID,
        filter=Nested(
                 scope=EQUIPMENT_VIEW_ID.as_property_ref("activity"), 
                 filter=In(ACTIVITY_VIEW_ID.as_property_ref('property1'), ["x", "y"])
               ),
        limit=None
        )

This works if the relation is a normal relation (i.e. going the other way from Activites to Equipment), but not from Equipment to Activities.

Secondly, because of this seeming limitation, are there benefits for having reverse direct relations?

You do the lookup on the activity and not the equipment. 

Something like this

client.data_modeling.instances.list( instance_type="node", sources=ACTIVITY_VIEW_ID, filter=In(ACTIVITY_VIEW_ID.as_property_ref('property1'), ["x", "y"]), limit=None )

Hi ​@Anders Albert ,

Apologies, I see now my question set up was incomplete.

Say, Equipment and Activities are defined as below: 

type Equipment
  @view(version: "1")
{
  property1: String
  activity: [Activities]
    @reverseDirectRelation(throughProperty: "equipment")
}


type Activities
  @view(version: "1")
{
  property2: String
  equipment: Equipment
}

What’s the best way to retrieve Equipment instances filtered on property1 (Equipment property) and property2 (Activities property)?

Something like this times out:

client.data_modeling.instances.list(
        instance_type="node",
        sources=EQUIPMENT_VIEW_ID,
        filter=And(
                 In(EQUIPMENT_VIEW_ID.as_property_ref("property1"), ["a", "b"],
                 Nested(
                 scope=EQUIPMENT_VIEW_ID.as_property_ref("activity"), 
                 filter=In(ACTIVITY_VIEW_ID.as_property_ref('property2'), ["x", "y"])
                 )
               ),
        limit=None
        )

I wrote up an article to try to address your issue. I think the issue is likely lack of indices on the properties you are filtering on, causing the timeout. Workaround is to try to split the query into two

I published it temporarily here: https://cognite-pygen.readthedocs-hosted.com/en/latest/docs-dev/query_filter.html