transformations-cli with FDM

Hi Niranjan,

Can you share with me your entire transformation config yaml?  Here or via Teams :).

Jason

Hi, it looks like you did not use the latest version of the cli, since we already renamed AlphaDMIDestinationConfig not DMIDestinationConfig , and the new type of FDM is InstancesDestinationConfig

Hi,

Thanks for your reply. I tried with latest transformations-cli (2.3.0) but I see the same error. I generated manifest file from CDF UI. Can you please provide me the example of manifest file with destination as FDM model? 

Thanks.

Hi,
Ah right you need to use the syntax with _ instead of camel case I think :)
https://github.com/cognitedata/transformations-cli/blob/main/tests/test_deploy.py#L185-L190

        destination:
            type: instances
            view_external_id: test_view
            view_version: test_view_version
            view_space_external_id: test_space
            instance_space_external_id: test_space

Hi,

Thanks for your reply. Looks like I am still missing something: 

transformations-cli deploy .

Deploying transformations...
Failed to parse transformation config, please check that you conform required fields and format: Invalid config: Wrong type for field "destination" - got "{'view_space_external_id': 'test-data-model-space', 'view_external_id': 'Entity', 'view_version': 2, 'instance_space_external_id': 'test-data-model-space', 'type': 'instances'}" of type dict instead of DestinationType, DestinationConfig, RawDestinationConfig, SequenceRowsDestinationConfig, DMIDestinationConfig, InstancesDestinationConfig, RawDestinationAlternativeConfig

Here is the entire contents of manifest file:

externalId: tr-cli-test-transformation
name: cli-test-transformation
query: >-
  file: transformation.sql
destination:
  type: instances
  view_space_external_id: test-data-model-space
  view_external_id: Test
  view_version: 0_2
  instance_space_external_id: test-data-model-space
ignoreNullFields: true
shared: true
action: upsert
authentication:
  clientId: ${COGNITE_CLIENT_ID}
  clientSecret: ${COGNITE_CLIENT_SECRET}
  tokenUrl: https://login.microsoftonline.com/${TENANT}/oauth2/v2.0/token
  # Optional: If idP requires providing the scopes
  cdfProjectName: ${CDF_PROJECT_NAME}
  scopes:
    - ${SCOPES}

transformations_cli  version is 2.3.0

Hi, I think it saw the view version as a number but not string according to your log
'view_version': 2

Can you force it to string by doing:
  view_version: “0_2”
In the manifest instead?

Hi, 

I tried that, now I see different error (unauthorized error), looks like some permission issue:

  return Instances(
Credentials for cli-test-transformation write failed to validate: Unauthorized | code: 401 

Hi,

I don’t think it’s permission issue (You would get 403 instead in that case). Here it’s 401 so you are not authorized, looks like your credentials is not correct. Can you double check it again, also is it possible to get the full error message, may be I can get a hint which endpoint returns that 401.

Hi,

I have verified the credentials again and I use the same credentials to call APIs

Here is the complete error:

> transformations-cli deploy .

Deploying transformations...
..\Python311\Lib\site-packages\cognite\transformations_cli\commands\deploy\transformations_api.py:95: UserWarning: Feature DataModelStorage is in beta and still in development. Breaking changes can happen in between patch versions.
  return Instances(
Credentials for cli-test-transformation write failed to validate: Unauthorized | code: 401 | X-Request-ID: xxxxx

Can you try to replace all env variable by raw string with the real value for the authentication section in the .yaml file to see?

authentication:
  clientId: "your ClientId"
  clientSecret: "yourClientSecret"
  tokenUrl: "https://login.microsoftonline.com/yourTenantId/oauth2/v2.0/token"
  scopes: 
    - "yourScope" # should be smthing like https://....
  cdfProjectName: "yourProjectName"

Aslo can you give me the request ID from the error above ? I can ask auth team to check the 401
401 | X-Request-ID: xxxxx
But I still think something is wrong with your credentials. If it’s a parsing value issue we should have seen it with other users who used the cli as well