Skip to main content
Solved

Help Deploying Hosted Extractors using Toolkit v0.3.12

  • November 28, 2024
  • 3 replies
  • 46 views

Forum|alt.badge.img+3

I am successfully deploying a Hosted Extractor, Source, Destination, Mapping and Job using the Cognite Toolkit v0.3.12.

 

The deployment succeeds and the hosted extractor appears in the UI, however it reports the following error when connecting to the source (from the Hosted Extractor Insight UI Tab):

Failed to read partitions: Management request failed: StatusError {code: StatusCode(401), description: Some("Attempted to perform an unauthorized operation.") }

 

I have provided the same connectivity information as we have working, in a manually-configured Hosted Extractor.  I am only trying to “package” this manually configured one into the toolkit, however there’s no feedback provided in the UI or the deployment suggesting why this is failing, and I need more logs to troubleshoot.

When setting up hosted extractors manually I find success pasting the connection string and allowing the UI to parse it into the correct fields.

My assumption is the SharedAccessKey is the problem with my deployment but there’s no guidance in the documentation to indicate whether it should be prefixed with `SharedAccessKey=<<secret>>` like in the connection string it works from within, so I’ve presumed not so, as adding the `sb://` prefix to the host failed to deploy.

I have confirmed the key works by, after deployment, re-pasting the connection string into deployed Source and re-saving.  This step completes the configuration, but this should be deployable and I’m not sure what’s missing.

 

One thought I'm not sure how to validate is whether the keyValue, which contains multiple forward slashes ( / ) and an equals sign ( = ) might be tripping up the toolkit parsing the value inside the yaml correctly, because the same value entered in the UI seems to work.

 

Our key value is being injected to the Source configuration YAML during the automated deployment using the toolkit, and the deployment pipeline reports the key is being loaded successfully.

Replacing only the Key value in the Source Connection UI once deployed, is enough to bring the hosted extractor online, and the value provided to do so is the same being injected by the deployment pipeline

 

Another test I’ve performed is pasting the connection string into the UI allowing it to populate the source, and then when inspecting the Key Value element inside the DOM before saving, I assert the value being provided is equals the value we’re injecting from the pipeline, so I think something’s maybe buggy with this version of the toolkit?

 

Additional thoughts on how to troubleshoot are appreciated.

Best answer by Jesse Jenken

Hi ​@Mithila Jayalath,

 

With a helping hand from ​@Jon Austin I think we’ve overcome this issue.

 

The way in which we’re injecting our Pipeline Variables into the environment was missing a mapping step (my fault) in the deployment template to map the new variable.

 

Everything now works and I’d like to compliment both the YAML reference library and API Documentation for helping fill in all of the questions I had when creating these configurations!

View original
Did this topic help you find an answer to your question?

3 replies

Mithila Jayalath
Seasoned Practitioner
Forum|alt.badge.img

@Jesse Jenken I’ll check on this and get back to you with an update.


Forum|alt.badge.img+3
  • Author
  • Seasoned
  • 12 replies
  • Answer
  • December 2, 2024

Hi ​@Mithila Jayalath,

 

With a helping hand from ​@Jon Austin I think we’ve overcome this issue.

 

The way in which we’re injecting our Pipeline Variables into the environment was missing a mapping step (my fault) in the deployment template to map the new variable.

 

Everything now works and I’d like to compliment both the YAML reference library and API Documentation for helping fill in all of the questions I had when creating these configurations!


Mithila Jayalath
Seasoned Practitioner
Forum|alt.badge.img

@Jesse Jenken thank you for the update and the feedback. I’m happy to hear that your issue is resolved.


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings