Skip to main content
Question

How cdf toolkit resolve dependencies between views ?

M
Seasoned

Hello, 

We are using cdf toolkit to deploy our container, views and data models.

We recently added many new views, these views are dependent. i.e: they reference each other in the properties through a source block or via implements block. Here an examples of these view dependencies:

- externalId: View1
  implements: []
  name: View1
  properties:
    test:
      container:
        externalId: Container1
        space: some_space
        type: container
      containerPropertyIdentifier: Test
      name: test
      source:
        externalId: View2
        space: some_space
        type: view
        version: v1
  space: some_space
  version: v1

- externalId: View2
  implements: []
  name: View2
  properties:
    wow:
      container:
        externalId: Container2
        space: some_space
        type: container
      containerPropertyIdentifier: Test
      name: wow
  space: some_space
  version: v1

or 

- externalId: Country
  implements:
    - externalId: CountryAttributes
      space: '{{sp_dm_dap_knowledge_graph}}_wv'
      type: view
      version: '{{dap_version}}'
  name: Country
  properties:
    Wells:
      connectionType: multi_reverse_direct_relation
      name: Wells
      source:
        externalId: Well
        space: '{{sp_dm_dap_knowledge_graph}}_wv'
        type: view
        version: '{{dap_version}}'
      through:
        identifier: Country
        source:
          externalId: Well
          space: '{{sp_dm_dap_knowledge_graph}}_wv'
          type: view
          version: '{{dap_version}}'
  space: '{{sp_dm_dap_knowledge_graph}}_wv'
  version: '{{dap_version}}'

- externalId: CountryAttributes
  implements: []
  name: CountryAttributes
  properties:
    CountryName:
      container:
        externalId: Country
        space: '{{sp_dm_dap_knowledge_graph}}'
        type: container
      containerPropertyIdentifier: CountryName
      name: CountryName
    RegionalEntity:
      container:
        externalId: Country
        space: '{{sp_dm_dap_knowledge_graph}}'
        type: container
      containerPropertyIdentifier: RegionalEntity
      name: RegionalEntity
  space: '{{sp_dm_dap_knowledge_graph}}_wv'
  version: '{{dap_version}}'


We have these errors: 

 

ERROR (ResourceCreationError): Failed to create resource(s). Error: Cannot upsert view 'sp_dm_dap_knowledge_graph_wv:Country/0.0.28'. Implemented view 'sp_dm_dap_knowledge_graph_wv:CountryAttributes/0.0.28' does not exist. | code: 400 | X-Request-ID: 027470c1-3679-9e04-a540-68ab60d31be8 | cluster: westeurope-1
The API Failed to process some items.
Successful (2xx): []
Unknown (5xx): []
Failed (4xx):

 

IMHO, this is issue in CDF toolkit dependencies resolutions. How CDF toolkit does this? How to avoid this errors? 

 

Thank you 

3 replies

Anders Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Anders Albert
  • Seasoned Practitioner
  • 106 replies
  • March 26, 2025

Just so I understand you correctly, you are trying to deploy multiple views which have interdependencies with CogniteToolkit? 

 

Some context here. There is a bug in the API that is on process of being fixed. CogniteToolkit has implemented a workaround from version 0.4.11, that does some analysis of the view dependencies and deploys them sequentially. Thus if you are on an earlier version than 0.4.11, I suggest you upgrade. 

Mithila Jayalath
1 person likes this
  • Mithila Jayalath
    Mithila Jayalath
M
Seasoned
  • Marwen TALEB
  • Author
  • Seasoned
  • 14 replies
  • March 27, 2025

Thanks ​@Anders Albert , this seems to resolve our issue. However we notice another one when deploying files. 

In fact we have this error:

 

Traceback (most recent call last):
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/bin/cdf-tk", line 8, in <module>
    sys.exit(app())
             ~~~^^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf.py", line 116, in app
    _app()
    ~~~~^^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/typer/main.py", line 339, in __call__
    raise e
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/typer/main.py", line 322, in __call__
    return get_command(self)(*args, **kwargs)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/typer/core.py", line 740, in main
    return _main(
        self,
    ...<6 lines>...
        **extra,
    )
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/typer/core.py", line 195, in _main
    rv = self.invoke(ctx)
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/typer/main.py", line 697, in wrapper
    return callback(**use_params)
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf_tk/apps/_core_app.py", line 280, in deploy
    cmd.run(
    ~~~~~~~^
        lambda: cmd.execute(
        ^^^^^^^^^^^^^^^^^^^^
    ...<9 lines>...
        )
        ^
    )
    ^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf_tk/commands/_base.py", line 55, in run
    raise e
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf_tk/commands/_base.py", line 52, in run
    result = execute(*args, **kwargs)
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf_tk/apps/_core_app.py", line 281, in <lambda>
    lambda: cmd.execute(
            ~~~~~~~~~~~^
        env_vars=env_vars,
        ^^^^^^^^^^^^^^^^^^
    ...<7 lines>...
        verbose=verbose,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf_tk/commands/deploy.py", line 165, in execute
    result = self.deploy_resources(
        loader_instance,
    ...<6 lines>...
        verbose=verbose,
    )
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf_tk/commands/deploy.py", line 197, in deploy_resources
    return self._deploy_resources(
           ~~~~~~~~~~~~~~~~~~~~~~^
        loader, env_vars, state.read_modules, dry_run, has_done_drop, has_dropped_data, force_update, verbose
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf_tk/commands/deploy.py", line 224, in _deploy_resources
    to_create, to_update, to_delete, unchanged, duplicated = worker.load_resources(
                                                             ~~~~~~~~~~~~~~~~~~~~~^
        files,
        ^^^^^^
    ...<3 lines>...
        verbose=verbose,
        ^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf_tk/loaders/_worker.py", line 165, in load_resources
    cdf_dict = self.loader.dump_resource(cdf_resource, local_dict)
  File "/Users/marwen.taleb/Library/Caches/pypoetry/virtualenvs/drillx-dwdap-cdf-toolkit-fxS2v_xU-py3.13/lib/python3.13/site-packages/cognite_toolkit/_cdf_tk/loaders/_resource_loaders/file_loader.py", line 146, in dump_resource
    if ds_id := dumped.pop("dataSetId"):

 
When trying to deploy this file conf : 
 


- dataSetExternalId: ds_streamlit_apps
  directory: /streamlit-apps/
  externalId: xxxxxxxx
  metadata:
    name: xxxxxx
    published: true
    theme: Light
  mimeType: application/json
  name: xxx.json
  source: Toolkit


 

I believe this is a bug since in file conf there is no dataSetId but dataSetExternalId as per your documentation.

 

Could you please take a look, thanks alot ! 

Anders Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Anders Albert
  • Seasoned Practitioner
  • 106 replies
  • March 28, 2025

You should never get a traceback like that, thus you are absolutely correct. Logged it as  bug, and will let you know once it is fixed.

M
1 person likes this
  • M
    Marwen TALEB

Reply


Terms & ConditionsCookie Settings

Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings