Cognite Connectors. queuing functionality

Hi,

We have built a Simulators Agent (Atlas AI Agent) with access to multiple Simulators Routines running through Cognite Connectors (PROSPER, GAP, and UniSim).

Today, our setup consists of a single Azure VM with one instance of each simulator and its corresponding Cognite Connector installed on this VM. The environment is not auto-scalable.

We have tested the queuing behavior with a single connector and observed positive results:

1. Submitting 10 requests for the same simulator routine with different inputs results in jobs being queued, executed sequentially, and returned successfully.
2. Launching the same routine simultaneously from 3 independent agent sessions also works without conflicts. Each session receives its results correctly.

Question

When we scale up to run thousands of simulations, we expect to deploy multiple instances of the same simulator and multiple Cognite Connectors of the same type.

Does Cognite provide any built-in queuing, load balancing, or workload distribution mechanism across multiple connectors of the same type?

We can see that queuing works with a single connector, but we would like to understand how requests are handled when multiple connectors are available.

Thanks,

Dmitriy Belozerov
Aker BP