Unable to Deploy IAM Groups: cognite.client.exceptions.CogniteAPIError: Failed to buffer the request body │ code: 400

We have encountered a scenario where we are unable to continue deploying to our production environment.

The deployment fails on the IAM groups which indicate an HTTP Error 400 with a note indicating the Cognite API Failed to bugger the request body.

Here is the important parts of the error: 

...
Deploying 19 iam.groups(resource_scoped) to CDF...
╭──────────────────────────────────────────────────────────────────────────────╮
│ Traceback (most recent call last):                                           │
│   File                                                                       │
│ "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/cognit │
│ e_toolkit/_cdf_tk/commands/deploy.py", line 423, in _update_resources        │
│     updated = loader.update(resources)                                       │
│               ^^^^^^^^^^^^^^^^^^^^^^^^                                       │
│   File                                                                       │
│ "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/cognit │
│ e_toolkit/_cdf_tk/loaders/_resource_loaders/auth_loaders.py", line 402, in   │
│ update                                                                       │
│     return self._upsert(items)                                               │
│            ^^^^^^^^^^^^^^^^^^^                                               │
│   File                                                                       │
│ "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/cognit │
│ e_toolkit/_cdf_tk/loaders/_resource_loaders/auth_loaders.py", line 395, in   │
│ _upsert                                                                      │
│     created = self.client.iam.groups.create(changed)                         │
│               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                         │
│   File                                                                       │
│ "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/cognit │
│ e/client/_api/iam.py", line 408, in create                                   │
│     return self._create_multiple(                                            │
│            ^^^^^^^^^^^^^^^^^^^^^^                                            │
│   File                                                                       │
│ "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/cognit │
│ e/client/_api_client.py", line 949, in _create_multiple                      │
│     summary.raise_compound_exception_if_failed_tasks(                        │
│   File                                                                       │
│ "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/cognit │
│ e/client/utils/_concurrency.py", line 77, in                                 │
│ raise_compound_exception_if_failed_tasks                                     │
│     self._raise_basic_api_error(str_format_element_fn, **task_lists)         │
│   File                                                                       │
│ "/opt/hostedtoolcache/Python/3.11.11/x64/lib/python3.11/site-packages/cognit │
│ e/client/utils/_concurrency.py", line 104, in _raise_basic_api_error         │
│     raise CogniteAPIError(                                                   │
│ cognite.client.exceptions.CogniteAPIError: Failed to buffer the request body │
│ | code: 400 | X-Request-ID: 7481428e-608f-9fd7-9ed4-da06200cd69d | cluster:  │
│ az-eastus-1                                                                  │
│ The API Failed to process some items.                                        │
│ Successful (2xx): []                                                         │
│ Unknown (5xx): []                                                            │
│ Failed (4xx): [{'name':
....

Following the ‘name’ key are all of my IAM groups.

We have overcome this issue by disabling a module that contains all of the IAM groups for one of our locations.

We believe the YAML is not the issue, as we can deploy one site, or the other site, but not both together.

This error feels like it’s trying to pack all of our IAM groups into a single API Call, which is overflowing the API buffer.

We have disabled the one site of IAM groups from deploying but this presents an issue for scaling our implementation out as we have several other sites and each site will have a number of groups required.

Our current Cognite Toolkit version is 0.3.12.