Atlas AI Query Tool: Resolve graph relations for deterministic agent evaluation

Hello :) 

Use case

As an AI evaluation engineer, I want graph relations returned by the agent to be resolved into the underlying equipment and time series instance IDs in the structured JSON output, so that recall metrics reflect the instances the agent actually identified, not only the relation objects it retrieved.

Background

I am evaluating agents that retrieve process data from an industrial knowledge graph in Cognite Data Fusion. I use a simple Python-based recall evaluator that compares expected equipment/time series IDs against IDs returned in the agent’s structured JSON output.

Current behavior

The agent may identify the correct equipment or time series in the natural-language response, but the structured JSON sometimes only contains the relation objects it used to infer them. This is mostly invisible to the end user, because the answer can still be correct. The issue is automated evaluation: correct retrievals may be undercounted because the expected equipment/time series IDs are missing from the JSON. I am spending too much time prompt engineering around this. The agent can produce the desired JSON structure sometimes, but not in a stable or deterministic manner.

Desired behavior

When graph relations are returned, the agent should perform one or more final listInstances calls to resolve/dereference them into the underlying equipment and time series instance IDs. This would make the structured JSON reflect the natural-language response, enabling faster, cheaper, and more deterministic recall evaluation for quality control and governance.