Hi Niranjan,

Can you share with me your entire transformation config yaml?  Here or via Teams .

Jason

Hi, it looks like you did not use the latest version of the cli, since we already renamed AlphaDMIDestinationConfig not DMIDestinationConfig , and the new type of FDM is InstancesDestinationConfig

Hi,

Thanks for your reply. I tried with latest transformations-cli (2.3.0) but I see the same error. I generated manifest file from CDF UI. Can you please provide me the example of manifest file with destination as FDM model? 

Thanks.

Hi,
Ah right you need to use the syntax with _ instead of camel case I think :)
https://github.com/cognitedata/transformations-cli/blob/main/tests/test_deploy.py#L185-L190

        destination:
            type: instances
            view_external_id: test_view
            view_version: test_view_version
            view_space_external_id: test_space
            instance_space_external_id: test_space

Hi,

Thanks for your reply. Looks like I am still missing something: 

transformations-cli deploy .

Deploying transformations...
Failed to parse transformation config, please check that you conform required fields and format: Invalid config: Wrong type for field "destination" - got "{'view_space_external_id': 'test-data-model-space', 'view_external_id': 'Entity', 'view_version': 2, 'instance_space_external_id': 'test-data-model-space', 'type': 'instances'}" of type dict instead of DestinationType, DestinationConfig, RawDestinationConfig, SequenceRowsDestinationConfig, DMIDestinationConfig, InstancesDestinationConfig, RawDestinationAlternativeConfig

Here is the entire contents of manifest file:

externalId: tr-cli-test-transformation
name: cli-test-transformation
query: >-
  file: transformation.sql
destination:
  type: instances
  view_space_external_id: test-data-model-space
  view_external_id: Test
  view_version: 0_2
  instance_space_external_id: test-data-model-space
ignoreNullFields: true
shared: true
action: upsert
authentication:
  clientId: ${COGNITE_CLIENT_ID}
  clientSecret: ${COGNITE_CLIENT_SECRET}
  tokenUrl: https://login.microsoftonline.com/${TENANT}/oauth2/v2.0/token
  # Optional: If idP requires providing the scopes
  cdfProjectName: ${CDF_PROJECT_NAME}
  scopes:
    - ${SCOPES}

transformations_cli  version is 2.3.0

Hi, I think it saw the view version as a number but not string according to your log
'view_version': 2

Can you force it to string by doing:
  view_version: “0_2”
In the manifest instead?

Hi, 

I tried that, now I see different error (unauthorized error), looks like some permission issue:

  return Instances(
Credentials for cli-test-transformation write failed to validate: Unauthorized | code: 401 

Hi,

I don’t think it’s permission issue (You would get 403 instead in that case). Here it’s 401 so you are not authorized, looks like your credentials is not correct. Can you double check it again, also is it possible to get the full error message, may be I can get a hint which endpoint returns that 401.

Hi,

I have verified the credentials again and I use the same credentials to call APIs

Here is the complete error:

> transformations-cli deploy .

Deploying transformations...
..\Python311\Lib\site-packages\cognite\transformations_cli\commands\deploy\transformations_api.py:95: UserWarning: Feature DataModelStorage is in beta and still in development. Breaking changes can happen in between patch versions.
  return Instances(
Credentials for cli-test-transformation write failed to validate: Unauthorized | code: 401 | X-Request-ID: xxxxx

Can you try to replace all env variable by raw string with the real value for the authentication section in the .yaml file to see?

authentication:
  clientId: "your ClientId"
  clientSecret: "yourClientSecret"
  tokenUrl: "https://login.microsoftonline.com/yourTenantId/oauth2/v2.0/token"
  scopes: 
    - "yourScope" # should be smthing like https://....
  cdfProjectName: "yourProjectName"

Aslo can you give me the request ID from the error above ? I can ask auth team to check the 401
401 | X-Request-ID: xxxxx
But I still think something is wrong with your credentials. If it’s a parsing value issue we should have seen it with other users who used the cli as well

I tried that as well, but I still the same issue.

Can you give me the uuid string of the request Id?
Should be the one with xxxx in your error log.
I can try to see with auth team why that request is rejected

Hi,

Here is the request id : 099cefef-6bc9-98c0-8122-68076440353c

Hello Niranjan,
I created a working sample I hope helps. I created an FDM targeted transformation in Fusion (version as of the date I wrote this) and used the export CLI button in the upper right. This created the following manifest.  I updated authentication section with the credentials that are required when the transformation itself is going to run.  This auth

# Manifest file downloaded from fusion
externalId: tr-BSEEWell-Header-Asset-FDM-cli
name: BSEE Well Header Asset - FDM - CLI
query: >-
  select
    cast(`API_WELL_NUMBER` as STRING) as externalId,
    cast(`WELL_NAME` as STRING) as name,
    cast(`API_WELL_NUMBER` as STRING) as apiNumber
  from
    `bsee`.`well_header`;
destination:
  view:
    space: space-well-model-simple
    externalId: Well
    version: "1"
  instanceSpace: space-well-model-simple
  type: nodes
ignoreNullFields: true
shared: true
action: upsert
dataSetExternalId: ds-bsee

# this is authentication credentials for Fusion, not the transformation-cli
authentication:
  clientId: mycliendid
  clientSecret: myclientsecret
  tokenUrl: https://login.microsoftonline.com/mytenantid/oauth2/v2.0/token
  # Optional: If idP requires providing the scopes
  cdfProjectName: cdf_project
  scopes:
   - https://api.cognitedata.com/.default

Regarding cognite-transformation-cli itself:

• I”m using a virtual python environment (version 3.10.5)
• I ran pip install cognite-transformations-cli in the above environment and confirmed that the version (transformations-cli --version) of the cli is  2.3.3
• I sourced the following shell script my path:

  #!/bin/sh
  
  export TRANSFORMATIONS_CLIENT_ID=myclientid
  export TRANSFORMATIONS_CLIENT_SECRET=myclientsecret
  export TRANSFORMATIONS_TOKEN_URL=https://login.microsoftonline.com/mytentantid/oauth2/v2.0/token
  export TRANSFORMATIONS_PROJECT=mycdfproject
  export TRANSFORMATIONS_CLUSTER=api
  export TRANSFORMATIONS_SCOPES=https://api.cognitedata.com/.default

• Run env at the command line to ensure you have the correct environment variables set:
• Run transformations-cli list.  This will list all transformations in the target project and will confirm you are correctly authenticated.
• Lastly, run transformations-cli deploy ./my-transformations to deploy your transformations to CDF.  The my-transformation folder contains the .yaml file containing the manifest above.
• Confirm the transformation has been deployed to your CDF project.

Hope this helps!

Jason

Hi @Niranjan Madhukar Karvekar,

Please let us know if you are able to try out Jason’s solution and if you are able to solve the issue. 

Br,
Dilini 

Hi @Niranjan Madhukar Karvekar,

I hope Jason’s reply has helped you to solve the issue. As of now, I will close this thread. Please feel free to create a new post if you still experience problems.

Br,
Dilini

Hi @Dilini Fernando

We have tried the steps mentioned in the Jason’s reply yet getting 401 error.

Sharing X-Request-ID for reference.
 

 Unauthorized | code: 401 | X-Request-ID: 2fc9c103-6de0-9ae2-bb38-f65110f66fed

Regards,

Sonali