Skip to main content
Seasoned ⭐️⭐️
belaz19
Seasoned ⭐️⭐️
September 4, 2026
Question

Cognite Connectors. queuing functionality

  • September 4, 2026
  • 1 reply
  • 49 views

Hi,

We have built a Simulators Agent (Atlas AI Agent) with access to multiple Simulators Routines running through Cognite Connectors (PROSPER, GAP, and UniSim).

Today, our setup consists of a single Azure VM with one instance of each simulator and its corresponding Cognite Connector installed on this VM. The environment is not auto-scalable.

We have tested the queuing behavior with a single connector and observed positive results:

  1. Submitting 10 requests for the same simulator routine with different inputs results in jobs being queued, executed sequentially, and returned successfully.
  2. Launching the same routine simultaneously from 3 independent agent sessions also works without conflicts. Each session receives its results correctly.

Question

When we scale up to run thousands of simulations, we expect to deploy multiple instances of the same simulator and multiple Cognite Connectors of the same type.

Does Cognite provide any built-in queuing, load balancing, or workload distribution mechanism across multiple connectors of the same type?

We can see that queuing works with a single connector, but we would like to understand how requests are handled when multiple connectors are available.

Thanks,

Dmitriy Belozerov
Aker BP

1 reply

vikram.chawan
Practitioner ⭐️⭐️⭐️
vikram.chawan
Practitioner ⭐️⭐️⭐️
September 7, 2026

Hi Dmitriy,

Thanks for the detailed writeup, the testing you've already done on single-connector queuing is exactly what we'd expect, and it's a good foundation for the scaling question.

Short answer: yes, Cognite has a built-in mechanism for this but it needs to be explicitly enabled. Here's the detail.

 

How job distribution works today (default behaviour)

By default, each simulation routine is statically bound to one specific connector instance at creation time. So if you deploy multiple PROSPER connectors on multiple VMs without further configuration, jobs targeted at Connector A will only ever be picked up by Connector A. Others would sit idle unless you manually split your routines across connector names. This explains why queuing "just works" today: you only have one instance of each connector, so there's no ambiguity about which one gets the job.

 

The built-in load-balancing feature

Cognite has shipped a proper load-balancing mechanism for exactly this scenario: multiple connector instances of the same simulator type, sharing a common pool of queued jobs. Under the hood, it uses an atomic "claim" operation on the backend. Distribution is first-come-first-served across connectors so it works naturally as an organic load balancer: idle or faster connectors pick up proportionally more work, with no manual sharding required on your end.

 

What's required to enable it

This feature is currently opt-in and off by default, and requires four things to be in place:

  • Connector version - GAP ≥ 2.0.0-beta-7 and PROSPER ≥ 2.1.0-beta-8.
  • Connector configuration - simulation-run-load-balancing-enabled: true set in each connector instance's config file (optionally tuning simulation-run-poll-limit, which controls how many runs a connector claims per poll).
  • A project-level feature toggle - this needs to be enabled on our side for your CDF project.
  • Routine configuration - routines must be created without a fixed simulator_integration_external_id (i.e., with queue=true), so runs are queued for any available connector rather than pinned to one. Existing statically-assigned routines won't pick up load balancing automatically.

More details can be found in the SDK docs (https://cognite-sdk-python.readthedocs-hosted.com/en/latest/simulators.html#) and cognite docs (https://docs.cognite.com/cdf/integration/guides/simulators/simulator_trigger#with-simulation-run-load-balancing).

    Terms & ConditionsAccessibility statement