Skip to main content
Seasoned ⭐️⭐️
belaz19
Seasoned ⭐️⭐️
September 4, 2026
Question

Cognite Connectors. queuing functionality

  • September 4, 2026
  • 2 replies
  • 80 views

Hi,

We have built a Simulators Agent (Atlas AI Agent) with access to multiple Simulators Routines running through Cognite Connectors (PROSPER, GAP, and UniSim).

Today, our setup consists of a single Azure VM with one instance of each simulator and its corresponding Cognite Connector installed on this VM. The environment is not auto-scalable.

We have tested the queuing behavior with a single connector and observed positive results:

  1. Submitting 10 requests for the same simulator routine with different inputs results in jobs being queued, executed sequentially, and returned successfully.
  2. Launching the same routine simultaneously from 3 independent agent sessions also works without conflicts. Each session receives its results correctly.

Question

When we scale up to run thousands of simulations, we expect to deploy multiple instances of the same simulator and multiple Cognite Connectors of the same type.

Does Cognite provide any built-in queuing, load balancing, or workload distribution mechanism across multiple connectors of the same type?

We can see that queuing works with a single connector, but we would like to understand how requests are handled when multiple connectors are available.

Thanks,

Dmitriy Belozerov
Aker BP

    2 replies

    vikram.chawan
    Practitioner ⭐️⭐️⭐️
    vikram.chawan
    Practitioner ⭐️⭐️⭐️
    September 7, 2026

    Hi Dmitriy,

    Thanks for the detailed writeup, the testing you've already done on single-connector queuing is exactly what we'd expect, and it's a good foundation for the scaling question.

    Short answer: yes, Cognite has a built-in mechanism for this but it needs to be explicitly enabled. Here's the detail.

     

    How job distribution works today (default behaviour)

    By default, each simulation routine is statically bound to one specific connector instance at creation time. So if you deploy multiple PROSPER connectors on multiple VMs without further configuration, jobs targeted at Connector A will only ever be picked up by Connector A. Others would sit idle unless you manually split your routines across connector names. This explains why queuing "just works" today: you only have one instance of each connector, so there's no ambiguity about which one gets the job.

     

    The built-in load-balancing feature

    Cognite has shipped a proper load-balancing mechanism for exactly this scenario: multiple connector instances of the same simulator type, sharing a common pool of queued jobs. Under the hood, it uses an atomic "claim" operation on the backend. Distribution is first-come-first-served across connectors so it works naturally as an organic load balancer: idle or faster connectors pick up proportionally more work, with no manual sharding required on your end.

     

    What's required to enable it

    This feature is currently opt-in and off by default, and requires four things to be in place:

    • Connector version - GAP ≥ 2.0.0-beta-7 and PROSPER ≥ 2.1.0-beta-8.
    • Connector configuration - simulation-run-load-balancing-enabled: true set in each connector instance's config file (optionally tuning simulation-run-poll-limit, which controls how many runs a connector claims per poll).
    • A project-level feature toggle - this needs to be enabled on our side for your CDF project.
    • Routine configuration - routines must be created without a fixed simulator_integration_external_id (i.e., with queue=true), so runs are queued for any available connector rather than pinned to one. Existing statically-assigned routines won't pick up load balancing automatically.

    More details can be found in the SDK docs (https://cognite-sdk-python.readthedocs-hosted.com/en/latest/simulators.html#) and cognite docs (https://docs.cognite.com/cdf/integration/guides/simulators/simulator_trigger#with-simulation-run-load-balancing).

      Committed ⭐️⭐️
      ajaypathak
      Committed ⭐️⭐️
      September 22, 2026

      Hi ​@vikram.chawan ,

      Thanks for the detailed explanation on the load-balancing feature — that clarifies the mechanism a lot.

      Follow-up on adoption: we've already built out a significant number of simulator routines and a fair amount of surrounding integration code (triggering runs via the API/SDK,). Today these are the "default behaviour" style — routines statically bound to specific connector instances.

      Since you mentioned that existing statically-assigned routines won't pick up load balancing automatically, I'd like to understand the migration effort before we commit:

      1. Routines — For our existing routines, is there an in-place way to switch them to queued (no fixed simulator_integration_external_id), or do we need to recreate them? If recreation is required, is there a recommended approach that preserves routine revision history, schedules, and external IDs so downstream references don't break?

      2. Code / API & SDK — Beyond setting queue=true on run creation, are there any other changes we need to make in how we trigger runs or read results/status? For example, does anything change in the run status lifecycle (queued → ready → running) that our polling/handling logic should account for?

      3. Schedules — How do scheduled triggers behave once load balancing is enabled? Anything we should adjust there?

      4. Migration path & recommendations — Does Cognite have an official recommended migration path, checklist, or tooling (e.g. CDF Toolkit / SDK scripts) for moving an existing, statically-assigned setup over to load balancing? Is a phased rollout (some routines queued, some still pinned) supported while we transition?

      5. Rollback — If we enable it and hit issues during preview, what's the recommended way to revert to the current static behaviour?

      Also the documentation says Simulation run load balancing (preview) so can we use this feature ? 

      Terms & ConditionsAccessibility statement