Skip to main content
Question

graphql and cdf_nodes doesn't work any more in big dataset

  • November 10, 2025
  • 2 replies
  • 49 views

We are now planning to integrate a large amount of historical data. The context about historical data and our data organization is in this topic:

We create an interface WellState, which contains 2 properties: WellId and Archived, in order to propagate the Archived flag to every object in our data models.

After integrated large historical data in our development env, we found that DataFetchingException always occurred in Daily(550K lines) and Operation(2M lines) in using GraphQL. And if using cdf_nodes to load Daily and Operation, the transformation will be blocked util that your session is closed.


Info: Daily has many relationships and Operation has only a relationship with Daily. But We have also other 3 objects like Operation(big dataset and a relationship with Daily), they can work well. 

GraphQL:

query MyQuery {
listOperation {
edges {
node {
externalId
space
}
}
}
listDaily {
edges {
node {
externalId
space
}
}
}
}

the result is always like that:

{
"errors": [
{
"message": "Client request(POST https://westeurope-1.cognitedata.com/api/v1/projects/totalenergies-sandbox/models/instances/query) invalid: 408 Request Timeout. Text: \"{\n \"error\": {\n \"code\": 408,\n \"message\": \"Graph query timed out. Reduce load or contention, or optimise your query.\",\n \"notices\": [\n {\n \"code\": \"significantHasDataFiltering\",\n \"category\": \"filtering\",\n \"level\": \"warning\",\n \"hint\": \"The provided `hasData` filters expand to several joins, which can be problematic depending on data distribution. Consider `requires` constraints, which can enable query time join pruning.\",\n \"grade\": \"C\",\n \"resultExpression\": \"0\",\n \"containers\": [\n {\n \"type\": \"container\",\n \"space\": \"cdf_cdm\",\n \"externalId\": \"CogniteDescribable\"\n },\n {\n \"type\": \"container\",\n \"space\": \"sp_dm_dap_knowledge_graph\",\n \"externalId\": \"Daily\"\n },\n {\n \"type\": \"container\",\n \"space\": \"sp_dm_dap_knowledge_graph\",\n \"externalId\": \"WellState\"\n }\n ]\n }\n ]\n }\n}\"",
"locations": [
{
"line": 10,
"column": 3
}
],
"path": [
"listDaily"
],
"extensions": {
"classification": "DataFetchingException"
}
},
{
"message": "Client request(POST https://westeurope-1.cognitedata.com/api/v1/projects/totalenergies-sandbox/models/instances/query) invalid: 408 Request Timeout. Text: \"{\n \"error\": {\n \"code\": 408,\n \"message\": \"Graph query timed out. Reduce load or contention, or optimise your query.\",\n \"notices\": [\n {\n \"code\": \"significantHasDataFiltering\",\n \"category\": \"filtering\",\n \"level\": \"warning\",\n \"hint\": \"The provided `hasData` filters expand to several joins, which can be problematic depending on data distribution. Consider `requires` constraints, which can enable query time join pruning.\",\n \"grade\": \"C\",\n \"resultExpression\": \"0\",\n \"containers\": [\n {\n \"type\": \"container\",\n \"space\": \"cdf_cdm\",\n \"externalId\": \"CogniteDescribable\"\n },\n {\n \"type\": \"container\",\n \"space\": \"sp_dm_dap_knowledge_graph\",\n \"externalId\": \"Operation\"\n },\n {\n \"type\": \"container\",\n \"space\": \"sp_dm_dap_knowledge_graph\",\n \"externalId\": \"WellState\"\n }\n ]\n }\n ]\n }\n}\"",
"locations": [
{
"line": 2,
"column": 3
}
],
"path": [
"listOperation"
],
"extensions": {
"classification": "DataFetchingException"
}
}
],
"data": {
"listOperation": null,
"listDaily": null
},
"extensions": {
"dmsQuery": {
"listOperation": {
"with": {
"0": {
"limit": 50,
"nodes": {
"filter": {
"and": [
{
"matchAll": {}
},
{
"hasData": [
{
"type": "view",
"space": "sp_dm_dap_knowledge_graph",
"externalId": "Operation",
"version": "0.72"
}
]
}
]
}
}
}
},
"select": {
"0": {
"sources": [
{
"source": {
"type": "view",
"space": "sp_dm_dap_knowledge_graph",
"externalId": "Operation",
"version": "0.72"
},
"properties": [
"Phase",
"Daily"
]
}
]
}
},
"includeTyping": true,
"forceCursorsDespitePerformanceHazard": true,
"debug": {}
},
"listDaily": {
"with": {
"0": {
"limit": 50,
"nodes": {
"filter": {
"and": [
{
"matchAll": {}
},
{
"hasData": [
{
"type": "view",
"space": "sp_dm_dap_knowledge_graph",
"externalId": "Daily",
"version": "0.72"
}
]
}
]
}
}
}
},
"select": {
"0": {
"sources": [
{
"source": {
"type": "view",
"space": "sp_dm_dap_knowledge_graph",
"externalId": "Daily",
"version": "0.72"
},
"properties": [
"Phase",
"Activity"
]
}
]
}
},
"includeTyping": true,
"forceCursorsDespitePerformanceHazard": true,
"debug": {}
}
},
"dmsDebug": {}
}
}

 

2 replies

  • Author
  • Committed
  • November 10, 2025

I tried to recover the Daily and Operation dataset in delete-recreate instances. After that, Daily can be sometimes visited in Graphql and cdf_nodes can load daily in transformations but Operation still doesn’t work at all.

How can I know what happened in this DataFetchingException ?


Arild  Eide
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • November 10, 2025

Hi ​@DSTDF 

It seems that both listOperation and listDaily are timing out and the GraphQL. The hints suggest that the time-out was caused by a join operation in Postgres that could potentially be optimized by adding a requires constraint. Please see this section in the article on query performance. Constraints are managed in the containers API.

 

If you created WellState for the single purpose of adding the Archived attribute, you can might consider adding it to the existing container as this would possibly also remove at least on join. Feel free to share the overall schema for additional context.

 

Thanks

 

Arild Eide