Skip to main content

Hi,

I’m going through the Python SDK- CDF Transformation module in the  DATA ENGINEER BASICS - TRANSFORM AND CONTEXTUALIZE course.

While I’m able to create the Transformation  object, I’m unable to run due to some Authentication issues. 

When I’m trying to run using the below code:

client.transformations.run(asset_transformation.id, wait=False)

I’m getting the following error :

Transformation job could not be created.
Error code: 403

API error: Invalid source/destination credentials: Could not authenticate with the OIDC credentials. Please check your credentials.

Request ID: 1a470c41-9001-989b-8f97-a2aaefdfd098

Is it due to some recent change in the Oidc credentials defined in the notebook currently?

 

Hi @Philips 

Could are able to authenticate in the first cell in the notebook?


Thanks for your feedback. 
It seems like an access right issue. 

Did you generate a secret and added it to the code? 
Did you encounter any errors earlier in the notebook, like in the first steps on the 
 

print(client.iam.token.inspect().projects)

And what information do you get via

print(client.iam.token.inspect())

 


Hi @Sofie Haug , @Gaetan Helness 

Thanks for the response.

Yes, I was able to authenticate the first cell in the notebook (i.e. I added the secret that i generated in the course).

When this statement is run,  print(client.iam.token.inspect().projects) I encountered no error.

Following the output I got:

>{ "url_name": "cdf-fundamentals", "groups": s 7233047623736400 ] }, { "url_name": "infield-training", "groups": s 1376085367190226 ] }, { "url_name": "publicdata", "groups": s 1701516184810448, 4860375157547584 ] }, { "url_name": "de-transformations", "groups": s 3974891759857484 ] }]

 

The information I get via below is attached.

print(client.iam.token.inspect())

best regards,

philips

 

 

 

 


It seems like an error with generating secrets from the course. @Rahul Kumar , could you look into it? 


Yes I agree with Roman

@Philips under the section Create credentials for use in Transformations, could you try to run this code?
This will try to authenticate with the client secret provided above in the notebook

from cognite.client.data_classes import OidcCredentials

creds = OidcCredentials(
client_id = CLIENT_ID,
client_secret = CLIENT_SECRET,
scopes = " ".join(SCOPES),
token_uri = f"{AUTHORITY_URI}/oauth2/v2.0/token",
cdf_project_name = COGNITE_PROJECT
)

cnf_secret = ClientConfig(
client_name="test-client",
base_url=f"https://{CDF_CLUSTER}.cognitedata.com",
project=COGNITE_PROJECT,
credentials=creds,
)
client_secret_auth = CogniteClient(cnf_secret)

print(client_secret_auth.iam.token.inspect())

 


The token inspect you printed is actually authenticated with your user, but great that works


Hi Gaetan, 

On running the code you gave, I’m getting below error.

AttributeError: 'OidcCredentials' object has no attribute 'authorization_header'

 


Hello Philips, 

Please generate the new secret in the course  and use newly generated secret for auth in the code  

Regards
Kumar 


You can then test the newly generated secret via

 

from cognite.client.credentials import OAuthClientCredentials

client_secret_auth = CogniteClient(ClientConfig(
client_name="test-client",
base_url=f"https://{CDF_CLUSTER}.cognitedata.com",
project=COGNITE_PROJECT,
credentials=OAuthClientCredentials(
token_url=f"{AUTHORITY_URI}/oauth2/v2.0/token",
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
scopes = " ".join(SCOPES)
),
))

print(client_secret_auth.iam.token.inspect())

 


@Philips let us know when you had a chance to test it out, but generating a new secret should fix the issue


You can then test the newly generated secret via

 

from cognite.client.credentials import OAuthClientCredentials

client_secret_auth = CogniteClient(ClientConfig(
client_name="test-client",
base_url=f"https://{CDF_CLUSTER}.cognitedata.com",
project=COGNITE_PROJECT,
credentials=OAuthClientCredentials(
token_url=f"{AUTHORITY_URI}/oauth2/v2.0/token",
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
scopes = " ".join(SCOPES)
),
))

print(client_secret_auth.iam.token.inspect())

 

@Gaetan Helness , The above code did run without any error.

But when I’m trying to run the transformation (given below), the error remains the same as before:

client.transformations.run(asset_transformation.id, wait=False)

 

Error:CogniteAPIError: Invalid source/destination credentials: Could not authenticate with the OIDC credentials. Please check your credentials. | code: 403 | X-Request-ID: 42d5a64d-d981-9154-8e9c-e302a2082e38


@Philips thanks, did you regenerate a secret, updated it in the code and reran through the notebook? 
What was the output of 

print(client_secret_auth.iam.token.inspect())

The problem is coming from the secret according to the error message. 

 


And if you can also send the output of
 

asset_transformation = client.transformations.retrieve_multiple(external_ids==f'{PREFIX}-assets']))0]
asset_transformation

 


@Philips thanks, did you regenerate a secret, updated it in the code and reran through the notebook? 
What was the output of 

print(client_secret_auth.iam.token.inspect())

The problem is coming from the secret according to the error message. 

 

Yes, I updated the secret in the code. The output is attached


And if you can also send the output of
 

asset_transformation = client.transformations.retrieve_multiple(external_ids==f'{PREFIX}-assets']))0]
asset_transformation

 

Output is attached


Thanks, everything looks quite good, it is hard to troubleshoot what is wrong. 
We will have another look through your attachments and get back to you


@Philips 
Could you complete the last step 4 Clean up resources and start from scratch, or at least delete the transformation

client.transformations.delete(external_id=df"{PREFIX}-assets", f"{PREFIX}-ts", f"{PREFIX}-dp"])

and recreate it. 
The previous secret that was not working has been stored in the transformation and therefore the transformation needs to be recreated


Yes, the cleaning up the transformation did the trick.

It is running fine now, 

Thanks a lot everyone for the prompt support!


Thanks for your patience and help for troubleshooting. 
Kudos to @roman.chesnokov , @Sofie Haug and @Rahul Kumar for the help as well


Reply