Getting AttributeError while using "bootstrap-cli prepare" command

Hi Basudeba,

thank you for reaching out for help on our Cognite Hub!

My name is Peter from Cognite and I wrote the `bootstrap-cli` tool.

The AttributeError states that bootstrap-cli expects in your provided config.yaml a `logger` section.

You can simply copy and add this `logger` section (from bootstrap-cli documentation) and I expect it to work.

logger:
  file:
    path: ./logs/test-deploy.log
    level: INFO
  console:
    level: INFO

Hopefully this is fixing your issue and you can proceed :)

best regards

Peter

(=PA=)

Hi Peter,

Thanks for looking into it, much appreciated.

As per the input, I have added “logger” section in the config yaml and with that experienced another AttributeError. I have also tried by adding “cognite” section along with “logger” section, and faced “access token” issue.

I am further exploring and looking into it. Will keep you posted by EOD if I figured out anything, otherwise I would like your assistance to get over this.

Error when added logger section without cognite section:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\BMandal2\bootstrap\inso-bootstrap-cli\incubator\bootstrap_cli\__main__.py", line 2365, in main
    bootstrap_cli()
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "C:\Users\BMandal2\bootstrap\inso-bootstrap-cli\incubator\bootstrap_cli\__main__.py", line 2252, in prepare
    BootstrapCore(config_file, command=CommandMode.PREPARE, debug=obj["debug"])
  File "C:\Users\BMandal2\bootstrap\inso-bootstrap-cli\incubator\bootstrap_cli\__main__.py", line 1377, in prepare
    _logger.debug(f"GROUPS in CDF:\n{self.deployed.groups}")
AttributeError: 'NoneType' object has no attribute 'groups'
 

Error when added logger section along with cognite section:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\BMandal2\bootstrap\inso-bootstrap-cli\incubator\bootstrap_cli\__main__.py", line 2365, in main
    bootstrap_cli()
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\click\decorators.py", line 38, in new_func
    return f(get_current_context().obj, *args, **kwargs)
  File "C:\Users\BMandal2\bootstrap\inso-bootstrap-cli\incubator\bootstrap_cli\__main__.py", line 2252, in prepare
    BootstrapCore(config_file, command=CommandMode.PREPARE, debug=obj["debug"])
  File "C:\Users\BMandal2\bootstrap\inso-bootstrap-cli\incubator\bootstrap_cli\__main__.py", line 525, in __init__
    self.client: CogniteClient = self.config.cognite.get_cognite_client(  # noqa
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\cognite\extractorutils\configtools.py", line 222, in get_cognite_client
    return CogniteClient(
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\cognite\client\_cognite_client.py", line 85, in __init__
    self._config = ClientConfig(
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\cognite\client\utils\_client_config.py", line 94, in __init__
    token_generator = utils._token_generator.TokenGenerator(
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\cognite\client\utils\_token_generator.py", line 21, in __init__
    self._generate_access_token()
  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\cognite\client\utils\_token_generator.py", line 49, in _generate_access_token
    raise CogniteAPIKeyError(
cognite.client.exceptions.CogniteAPIKeyError: Error generating access token: missing_token, 400, Missing access token parameter.
 

Hi Basudeba,

just for your confirmation:

• the `prepare` command requires to access Cognite with a service-account (an AAD App)
• which requires the `cognite` section available too
• as documented here
  • it is using a `.env` file to separate the secrets from the configuration

cognite:
  host: ${BOOTSTRAP_CDF_HOST}
  project: ${BOOTSTRAP_CDF_PROJECT}
  #
  # IdP login:
  #
  idp-authentication:
    client-id: ${BOOTSTRAP_IDP_CLIENT_ID}
    secret: ${BOOTSTRAP_IDP_CLIENT_SECRET}
    scopes:
      - ${BOOTSTRAP_IDP_SCOPES}
    token_url: ${BOOTSTRAP_IDP_TOKEN_URL}

Which requires -- beside others -- a `BOOTSTRAP_IDP_CLIENT_ID / BOOTSTRAP_IDP_CLIENT_SECRET` (client/secrets) of an AAD Application for service-level authentication.

• which you had to create first in your AAD tenant
• because this (first) AAD App is used for root access to Cognite, a typical name could be “CDF_DEV_ROOT_CLIENT” (“DEV” or “TEST”  or “PROD” in case SLB has staging projects connected to the same SLB AAD)
• which must be a member of the same AAD Group which was used requesting the CDF Project => “Admin group source ID *”

I hope this makes sense, and I’m happy to follow up and improve the documentation after our session.

Bets regards

Peter

(=PA=)

Thanks Peter!

I had missed .env file.  Now I can connect to the slb-pdf project. However I still see “API Failed to process some items” error with 403 unauthorized error code. As per the error code, It looks like the client id/secrets that I use does not have enough permission to perform the actions. 

Error Snippet:

Prepare CDF Project ...
2023-06-01 08:30:48.821 UTC [INFO    ] MainThread - Starting CDF Bootstrap configuration for command: <prepare>
2023-06-01 08:30:49.555 UTC [INFO    ] MainThread - Successful connection to CDF client to project: 'slb-pdf'
2023-06-01 08:30:50.114 UTC [INFO    ] MainThread - Deployed CDF Resource counts:
            RAW Dbs(n/a with this command)
            Data Sets(n/a with this command)
            CDF Groups(6)
Traceback (most recent call last):
  File "<string>", line 1, in <module>

  File "C:\Users\BMandal2\AppData\Local\pypoetry\Cache\virtualenvs\inso-bootstrap-cli-Jo_B4J-G-py3.10\lib\site-packages\cognite\client\utils\_concurrency.py", line 85, in collect_exc_info_and_raise
    raise CogniteAPIError(
cognite.client.exceptions.CogniteAPIError: Unauthorized | code: 403 | X-Request-ID: 94abaa61-3e37-93c4-9082-08c7d9a35207
The API Failed to process some items.
Successful (2xx): []
Unknown (5xx): []
Failed (4xx): [{'name': 'cdf:bootstrap', 'source_id': 'f2e5b1ef-e601-46ec-87ec-25f62810ad1f', 'capabilities': [{'datasetsAcl': {'actions': ['READ', 'WRITE', 'OWNER'], 'scope': {'all': {}}}}, {'rawAcl': {'actions': ['READ', 'WRITE', 'LIST'], 'scope': {'all': {}}}}, {'groupsAcl': {'actions': ['LIST', 'READ', 'CREATE', 'UPDATE', 'DELETE'], 'scope': {'all': {}}}}, {'projectsAcl': {'actions': ['READ', 'UPDATE'], 'scope': {'all': {}}}}]}]
 

I am using a set of client id/secrets in the .env file along with other environment variables. Do we need to provide additional roles/permissions so that It can perform the actions without 403 ?

Hi Basudeba,

> I had missed .env file.

Not mandatory, just the recommended practice not storing secrets in config files, which maybe get added to a version-control-system (like git)

The permission problem of your AAD App should not happen, as long as you have added the app to the same AAD Group which you used for requesting the CDF Project.

“Typically” the very first CDF Group setup and linked to this AAD Group is named “oidc-admin-group” in Cognite, and as long as your user has the permissions, you open it from “Manage > Access Management” select the group, go to edit and copy the “Source ID” (which is the AAD Group object-id)?

This (screenshot) are the required capabilities in Cognite, the AAD App requires (through its AAD Group > Cognite linking)

I’m sorry if this sounds over-complicated, but I try to provide enough information and context, hopefully to help you navigate and solving your problem ASAP.

Best regards

Peter

(=PA=)

@Basudeba Mandal ,
I can help administer the clientId and secret you need to to access slb-pdf.  I’ve reached out on Teams to verify and share with you the correct credentials.

Jason