Skip to main content
P
Seasoned ⭐️⭐️
Philips
Seasoned ⭐️⭐️
April 24, 2023
Solved

Unable to run the CDF Transformation job created

  • April 24, 2023
  • 19 replies
  • 298 views

Hi,

I’m going through the Python SDK- CDF Transformation module in the  DATA ENGINEER BASICS - TRANSFORM AND CONTEXTUALIZE course.

While I’m able to create the Transformation  object, I’m unable to run due to some Authentication issues. 

When I’m trying to run using the below code:

client.transformations.run(asset_transformation.id, wait=False)

I’m getting the following error :

Transformation job could not be created.
Error code: 403

API error: Invalid source/destination credentials: Could not authenticate with the OIDC credentials. Please check your credentials.

Request ID: 1a470c41-9001-989b-8f97-a2aaefdfd098

Is it due to some recent change in the Oidc credentials defined in the notebook currently?

 

    Best answer by Gaetan Helness

    @Philips 
    Could you complete the last step 4 Clean up resources and start from scratch, or at least delete the transformation

    client.transformations.delete(external_id=[f"{PREFIX}-assets", f"{PREFIX}-ts", f"{PREFIX}-dp"])

    and recreate it. 
    The previous secret that was not working has been stored in the transformation and therefore the transformation needs to be recreated

    19 replies

    Sofie Haug
    Expert ⭐️⭐️⭐️⭐️
    Sofie Haug
    Cognite Academy Instructor
    April 24, 2023

    Hi @Philips 

    Could are able to authenticate in the first cell in the notebook?

    Sofie | Cognite Academy
    Gaetan Helness
    MVP ⭐️⭐️⭐️⭐️⭐️
    Gaetan Helness
    MVP ⭐️⭐️⭐️⭐️⭐️
    April 24, 2023

    Thanks for your feedback. 
    It seems like an access right issue. 

    Did you generate a secret and added it to the code? 
    Did you encounter any errors earlier in the notebook, like in the first steps on the 
     

    print(client.iam.token.inspect().projects)

    And what information do you get via

    print(client.iam.token.inspect())

     

    P
    Seasoned ⭐️⭐️
    PhilipsAuthor
    Seasoned ⭐️⭐️
    April 25, 2023

    Hi @Sofie Haug , @Gaetan Helness 

    Thanks for the response.

    Yes, I was able to authenticate the first cell in the notebook (i.e. I added the secret that i generated in the course).

    When this statement is run,  print(client.iam.token.inspect().projects) I encountered no error.

    Following the output I got:

    [{ "url_name": "cdf-fundamentals", "groups": [ 7233047623736400 ] }, { "url_name": "infield-training", "groups": [ 1376085367190226 ] }, { "url_name": "publicdata", "groups": [ 1701516184810448, 4860375157547584 ] }, { "url_name": "de-transformations", "groups": [ 3974891759857484 ] }]

     

    The information I get via below is attached.

    print(client.iam.token.inspect())

    best regards,

    philips

     

     

     

     

    roman.chesnokov
    Expert ⭐️⭐️⭐️⭐️
    roman.chesnokov
    Expert ⭐️⭐️⭐️⭐️
    April 25, 2023

    It seems like an error with generating secrets from the course. @Rahul Kumar , could you look into it? 

    Gaetan Helness
    MVP ⭐️⭐️⭐️⭐️⭐️
    Gaetan Helness
    MVP ⭐️⭐️⭐️⭐️⭐️
    April 25, 2023

    Yes I agree with Roman

    @Philips under the section Create credentials for use in Transformations, could you try to run this code?
    This will try to authenticate with the client secret provided above in the notebook

    from cognite.client.data_classes import OidcCredentials

    creds = OidcCredentials(
            client_id = CLIENT_ID,
            client_secret = CLIENT_SECRET,
            scopes = " ".join(SCOPES),
            token_uri = f"{AUTHORITY_URI}/oauth2/v2.0/token",
            cdf_project_name = COGNITE_PROJECT
    )

    cnf_secret = ClientConfig(
        client_name="test-client",
        base_url=f"https://{CDF_CLUSTER}.cognitedata.com",
        project=COGNITE_PROJECT,
        credentials=creds,
    )
    client_secret_auth = CogniteClient(cnf_secret)

    print(client_secret_auth.iam.token.inspect())

     

    Gaetan Helness
    MVP ⭐️⭐️⭐️⭐️⭐️
    Gaetan Helness
    MVP ⭐️⭐️⭐️⭐️⭐️
    April 25, 2023

    The token inspect you printed is actually authenticated with your user, but great that works

    P
    Seasoned ⭐️⭐️
    PhilipsAuthor
    Seasoned ⭐️⭐️
    April 25, 2023

    Hi Gaetan, 

    On running the code you gave, I’m getting below error.

    AttributeError: 'OidcCredentials' object has no attribute 'authorization_header'

     

      R
      Practitioner ⭐️⭐️⭐️
      Rahul Kumar
      Cognite Academy Instructor
      April 25, 2023

      Hello Philips, 

      Please generate the new secret in the course  and use newly generated secret for auth in the code  

      Regards
      Kumar 

      Rahul Kumar
      Gaetan Helness
      MVP ⭐️⭐️⭐️⭐️⭐️
      Gaetan Helness
      MVP ⭐️⭐️⭐️⭐️⭐️
      April 25, 2023

      You can then test the newly generated secret via

       

      from cognite.client.credentials import OAuthClientCredentials

      client_secret_auth = CogniteClient(ClientConfig(
          client_name="test-client",
          base_url=f"https://{CDF_CLUSTER}.cognitedata.com",
          project=COGNITE_PROJECT,
          credentials=OAuthClientCredentials(
          token_url=f"{AUTHORITY_URI}/oauth2/v2.0/token",
          client_id=CLIENT_ID,
          client_secret=CLIENT_SECRET,
          scopes = " ".join(SCOPES)
      ),
      ))

      print(client_secret_auth.iam.token.inspect())

       

      Gaetan Helness
      MVP ⭐️⭐️⭐️⭐️⭐️
      Gaetan Helness
      MVP ⭐️⭐️⭐️⭐️⭐️
      April 25, 2023

      @Philips let us know when you had a chance to test it out, but generating a new secret should fix the issue

      Show more replies
      Terms & ConditionsAccessibility statement