Solved

FDM Graphql - Edges - Limitation over the number of edges fetched (only 50) when it is as a nested property

N
Rank
Userlevel 1

Hello,

We are trying to fetch the edges of an view in a graphql.

But graphql is returning only 50 edges at max. We see the cursor option in graphql, but it seems that it has been not implemented and is giving null even if there exists more data.

So question is - 

  1. When the cursor functionality for edges will be avaialble in graphql or some know issue as of now ?
  2. If this is a limitation how do we get edges data joined with the view data where the edge is present using the rest endpoints. Any best combination suggested as there will be multiple stages to resolve the cursor and the edges data, as we understand it.

Request your quick ideas on this how it could be achieved.

icon

Best answer by mattmurph9 30 June 2023, 13:35

View original

13 replies

M
Rank

You can pass a limit to the nested query like this

{
  listMovie {
    items {
      actors(first: 1000) {
        items {
          externalId
        }
      }
    }
  }
}

if you want to paginate on a nested level, you need to use a `get...ById` query instead of the list query. As of now we only support pagination on the first level in list queries, and on the second level in getById queries.

Dilini Fernando
Rank
Userlevel 4
Badge +2

Hi @Neerajkumar Bhatewara

We are following up to see whether you're satisfied with the responses you've received?

If you found the responses offered by our community members to be instructive and helpful, we kindly request that you consider marking the most helpful response as the "Best Answer."This little action will acknowledges the person who offered the answer and also makes it simpler for other community members to find the most correct comments.

Best regards,
Dilini

 

N
Rank
Userlevel 1

@mattmurph9 - Thanks for your reply.

By using the getByIds means we will have to first get the ids in a separate query and then pass the ids to this getByIds query and then we can use the nested cursor, correct ? Means two queries separately, if I understand correct!

 

M
Rank

The getById queries only support fetching a single item by id. Then you can paginate the children. Yes this means separate queries. Can you explain the use case though? Paginating children when multiple parents are returned gets quite confusing, and I’m not sure there’s any strong use cases beyond “give me all the data”

N
Rank
Userlevel 1

Hello mattmurph9
Consider below sample FDM Model.

Well{ id: String, name: String, region: String, wellTests : [WellTest] }
WellTest{ id: String, passFail: Boolean}

In above model Well
My scenario is I want all the wellTests performed for Wells say in region West.

So it seems this cannot be done from graphql due to cursor limitation and some WIP.
But we cannot stay blocked so meanwhile, is there any rest api that can give me above data in one call ? I was exploring instances/query api, not sure if that would be correct candidate for this usecase.

Hope this information helps to help us :)

M
Rank

“I want all the wellTests performed for Wells in region ‘West’”
with graphql, you could do

{
  listWell(filter: { region: { eq: "West" } }) {
    items {
      wellTests {
        items {
          id
          passFail
        }
      }
    }
  }
}

is this what you’re currently trying with graphql (leading to the cursor limitation you mention)? If so that’s a bug I will look into.

In terms of a workaround, the graphql layer is calling the `instances/query` endpoint, so everything possible with graphql is possible with that (and more). The body of that request would look something like this:
 

{
    "with": {
        "0": {
            "limit": 50,
            "nodes": {
                "filter": {
                    "and": [
                        {
                            "matchAll": {}
                        },
                        {
                            "hasData": [
                                {
                                    "type": "view",
                                    "space": "someSpace",
                                    "externalId": "Well",
                                    "version": "someVersion"
                                }
                            ]
                        },
                        {
                            "or": [
                                {
                                    "equals": {
                                        "property": [
                                            "someSpace",
                                            "Well/someVersion",
                                            "region"
                                        ],
                                        "value": "West"
                                    }
                                }
                            ]
                        }
                    ]
                }
            }
        },
        "0_edges_0": {
            "limit": 10000,
            "edges": {
                "from": "0",
                "maxDistance": 1,
                "direction": "outwards",
                "filter": {
                    "equals": {
                        "property": [
                            "edge",
                            "type"
                        ],
                        "value": {
                            "space": "someSpace",
                            "externalId": "Well.wellTests"
                        }
                    }
                },
                "limitEach": 50
            }
        },
        "0_0": {
            "limit": 10000,
            "nodes": {
                "from": "0_edges_0",
                "filter": {
                    "and": [
                        {
                            "matchAll": {}
                        },
                        {
                            "hasData": [
                                {
                                    "type": "view",
                                    "space": "someSpace",
                                    "externalId": "WellTest",
                                    "version": "someOtherVersion"
                                }
                            ]
                        }
                    ]
                }
            }
        }
    },
    "select": {
        "0_0": {
            "sources": [
                {
                    "source": {
                        "type": "view",
                        "space": "someSpace",
                        "externalId": "WellTest",
                        "version": "someOtherVersion"
                    },
                    "properties": [
                        "id",
                        "passFail"
                    ]
                }
            ]
        }
    }
}

You would need to replace space and view versions in this query to use it, and possibly make some tweaks, but that should at least get you started with the query endpoint. Let me know if there’s anything else you need!

N
Rank
Userlevel 1

Thanks @mattmurph9 for the reply.
It works fine when we have to just get the data of the end node ie WellTest, but we need few properties from start node(Well Asset) as well along with data of end Node.

I will have to get three different result sets and then join them based on the edge start node id and edge end node id.

Is there any way we directly get the merged data set directly from query endpoint ? In other words I want Well id and Well Name for all wells in West region along with WellTest ids and passFail result merged together with Well Details. Graphql output is merged one, is there anyway to do so with query endpoint as well?

M
Rank

Absolutely you can do this. The query would look similar, but adding another key in the select statement
 

{
    "with": {
        "0": {
            "limit": 50,
            "nodes": {
                "filter": {
                    "and": [
                        {
                            "matchAll": {}
                        },
                        {
                            "hasData": [
                                {
                                    "type": "view",
                                    "space": "someSpace",
                                    "externalId": "Well",
                                    "version": "someVersion"
                                }
                            ]
                        },
                        {
                            "or": [
                                {
                                    "equals": {
                                        "property": [
                                            "someSpace",
                                            "Well/someVersion",
                                            "region"
                                        ],
                                        "value": "West"
                                    }
                                }
                            ]
                        }
                    ]
                }
            }
        },
        "0_edges_0": {
            "limit": 10000,
            "edges": {
                "from": "0",
                "maxDistance": 1,
                "direction": "outwards",
                "filter": {
                    "equals": {
                        "property": [
                            "edge",
                            "type"
                        ],
                        "value": {
                            "space": "someSpace",
                            "externalId": "Well.wellTests"
                        }
                    }
                },
                "limitEach": 50
            }
        },
        "0_0": {
            "limit": 10000,
            "nodes": {
                "from": "0_edges_0",
                "filter": {
                    "and": [
                        {
                            "matchAll": {}
                        },
                        {
                            "hasData": [
                                {
                                    "type": "view",
                                    "space": "someSpace",
                                    "externalId": "WellTest",
                                    "version": "someOtherVersion"
                                }
                            ]
                        }
                    ]
                }
            }
        }
    },
    "select": {
        "0": {
            "sources": [
                {
                    "source": {
                        "type": "view",
                        "space": "someSpace",
                        "externalId": "Well",
                        "version": "someVersion"
                    },
                    "properties": [
                        "id",
                        "name"
                    ]
                }
            ]
        },
        "0_0": {
            "sources": [
                {
                    "source": {
                        "type": "view",
                        "space": "someSpace",
                        "externalId": "WellTest",
                        "version": "someOtherVersion"
                    },
                    "properties": [
                        "id",
                        "passFail"
                    ]
                }
            ]
        }
    }
}

 

N
Rank
Userlevel 1

Yes thats what I did, but with this you cannot connect the data from 0_0 with 0.
What we have to do is get the edges part as well and after iterating all the edges we get to know about the start nodes and end nodes and then we join the start nodes and end nodes data separately.

I was looking for something which can give me the connected data together in one structure. This is something probably a consumer would want. I believe that there has to be some way where the connected data can be retrieved somehow without the end user parse and connect the data(between start and end nodes).

Can we do this ?

N
Rank
Userlevel 1

THis is my query
 

{

    "with": {

        "AssetNodes": {

            "limit": 3,

            "nodes": {

                "filter": {

                    "and": [

                        {

                            "matchAll": {}

                        },

                        {

                            "hasData": [

                                {

                                    "type": "view",

                                    "space": "test-data-model-demo-space",

                                    "externalId": "Asset",

                                    "version": "532d19de4194f1"

                                }

                            ]

                        }

                    ]

                }

            }

        },

        "EdgesFromAssetToWellTest": {

            "limit": 10000,

            "edges": {

                "from": "AssetNodes",

                "maxDistance": 1,

                "direction": "outwards",

                "filter": {

                    "equals": {

                        "property": [

                            "edge",

                            "type"

                        ],

                        "value": {

                            "space": "test-data-model-demo-space",

                            "externalId": "Asset.WellTests"

                        }

                    }

                },

                "limitEach": 50

            }

        },

        "WellTestNodes": {

            "limit": 10000,

            "nodes": {

                "from": "EdgesFromAssetToWellTest",

                "chainTo" : "destination",

                "filter": {

                    "and": [

                        {

                            "matchAll": {}

                        },

                        {

                            "hasData": [

                                {

                                    "type": "view",

                                    "space": "test-data-model-demo-space",

                                    "externalId": "WellTest",

                                    "version": "baee32515982c5"

                                }

                            ]

                        }

                    ]

                }

            }

        }

    },

    "select": {

        "WellTestNodes": {

            "sources": [

                {

                    "source": {

                        "type": "view",

                        "space": "test-data-model-demo-space",

                        "externalId": "WellTest",

                        "version": "baee32515982c5"

                    },

                    "properties": [

                        "WellTestID",

                        "CompletionName"

                    ]

                },

                {

                    "source": {

                        "type": "view",

                        "space": "test-data-model-demo-space",

                        "externalId": "Asset",

                        "version": "532d19de4194f1"

                    },

                    "properties": [

                        "AssetId",

                        "AssetName"

                    ]

                }

            ]

        },

        "EdgesFromAssetToWellTest":{},

        "AssetNodes":

        {

            "sources": [

                {

                        "source": {

                            "type": "view",

                            "space": "test-data-model-demo-space",

                            "externalId": "Asset",

                            "version": "532d19de4194f1"

                        },

                        "properties": [

                            "AssetId",

                            "AssetName"

                        ]

                }

            ]

        }

    },

    "cursors": {

        "AssetNodes": "Z0FBQUFBQmt2bklGSTRrT2JDcS1YOTVrMGZrRk5oZXF2TW1VUXEwaGE0aTItaUZaeU5PNWRFd0VzV3drOUM2UUNMR2Q0VUVvSlkyTmhlc1prRjBzZ0RrMG5hNFJWU3M4LTUyOTNRRnhNQlJHY3NzeEVaOXNoMUlpZzNxOUN5Ukk1THZXQlQxQVNQVHptdEFWTVFLR1JWc2RBLVgtWDIxT3N2NGdLZDk5RnAxNWRwTVFxcjJnUjA4PQ=="

    }

}

If you see the part EdgesFromAssetToWellTest in payload we can add the properties in that part, I went through the documentation but did not get what can be properties of edges, is this something that can be used to get the connected data ?

M
Rank

If you want the data in a connected structure, you’d have to use graphql

N
Rank
Userlevel 1

Back to square one, we cannot iterate over internal elements or structure using graphql.

So I think we can say that there is no way to get the connected data anyhow from cognite system. Consumer needs to do it on its own by using the query endpoint or some other way.
 

 

PS - I am not sure how this got marked as solved, may be I marked it accidently and no option to reopen.

M
Rank

Ok so the query endpoint is off the table here if a requirement is to get a connected structure back.

Let’s try to figure out what you would require for the graphql to work. So currently with the `getById` graphql queries you can paginate through children. The problem here is that you don’t want to make multiple requests for multiple parent ids right?
 

With the `list` queries, we don’t support pagination of children, but you can get up to 1000 children. The problem here is that 1000 is not enough right?

If my above statements are true, can we talk about the use case more? Is this query for displaying data in a UI?

Reply


Terms & ConditionsCookie Settings