Skip to main content

Hi Team, I want to change the message for the custom extractor’s run history message by python sdk. Right now, message says “successful shutdown” after successfully running the extractor, but I want to change it with how many datapoints are uploading and in how much time.
I tried to do it by following methods:

  1. By using abstract class Extractor in our code to overwrite the _report_success method’s message.

       2.By using this method 

 cognite.extractorutils.util.add_extraction_pipeline(extraction_pipeline_ext_id: str, cognite_client: CogniteClient, heartbeat_waiting_time: int = 600, added_message: str = '')

       3. By using below class too:

cclient.data_classes.extractionpipelines.ExtractionPipeline(id: int | None = None, external_id: str | None = None, last_message: str | None = None)

 

 

There is no way of passing the custom message here in my knowledge if it’s there let me know!

And is there any way to pass logs in extraction pipeline?

@Shweta 

Each extraction run can have it’s own message: https://cognite-sdk-python.readthedocs-hosted.com/en/latest/data_ingestion.html#cognite.client._api.extractionpipelines.ExtractionPipelineRunsAPI.create

https://cognite-sdk-python.readthedocs-hosted.com/en/latest/data_ingestion.html#cognite.client.data_classes.extractionpipelines.ExtractionPipelineRunWrite
 

Is this what you need?

-Jason

Hi @Jason Dressel thank you so much for your reply but I can’t find  this class ExtractionPipelineRunWrite in data_classes library! that’s why I am unable to import it for my use. 
 

@Shweta what version of the python sdk are you using?  You many need to upgrade.
Try just using ExtractionPipelineRun

-Jason

       

Yes @Jason Dressel  upgraded the version and used the above classes like this and I am getting below error!
creds = OAuthClientCredentials(

        token_url=clientConfig.token_url, 

        client_id=clientConfig.client_id, 

        client_secret=clientConfig.client_secret, 

        scopes=clientConfig.scope

        )

 

    cnf = ClientConfig(client_name=clientConfig.project, project=clientConfig.project_name, credentials=creds, base_url=clientConfig.base_url)

    client = CogniteClient(cnf)


class usage:
    newSuccessRun = client.extraction_pipelines.create(ExtractionPipelineRunWrite( extpipe_external_id="your_ext_id",status="success", message="Error message1"))

 

and same below error with ExtractionPipelineRun.

Error:

TypeError: 'extraction_pipeline' must be one of types r<class 'cognite.client.data_classes.extractionpipelines.ExtractionPipelineCore'>, typing.Sequence], not <class 'cognite.client.data_classes.extractionpipelines.ExtractionPipelineRunWrite'>
 

can you help me with this?

@Shweta 
Sorry for the delay.  It appears there may have been a bug in the SDK that is now fixed.  Please try with the following code snippet:

run = ExtractionPipelineRunWrite(extpipe_external_id="csv:oee:datapoints", status="failure", message="custom error message")
result = client.extraction_pipelines.runs.create(run)
result

-Jason

Yes, thank you very much for you help, issue is resolved.

Reply


Terms & ConditionsCookie Settings