Enable logging to be accessible from Extraction Pipeline UI

  • 10 January 2024
  • 6 replies


Hi Team, I want to change the message for the custom extractor’s run history message by python sdk. Right now, message says “successful shutdown” after successfully running the extractor, but I want to change it with how many datapoints are uploading and in how much time.
I tried to do it by following methods:

  1. By using abstract class Extractor in our code to overwrite the _report_success method’s message.

       2.By using this method 

 cognite.extractorutils.util.add_extraction_pipeline(extraction_pipeline_ext_id: str, cognite_client: CogniteClient, heartbeat_waiting_time: int = 600, added_message: str = '')

       3. By using below class too:

cclient.data_classes.extractionpipelines.ExtractionPipeline(id: int | None = None, external_id: str | None = None, last_message: str | None = None)



There is no way of passing the custom message here in my knowledge if it’s there let me know!

And is there any way to pass logs in extraction pipeline?


Best answer by Jason Dressel 22 February 2024, 19:38

View original

6 replies

Userlevel 4


Each extraction run can have it’s own message:

Is this what you need?



Hi @Jason Dressel thank you so much for your reply but I can’t find  this class ExtractionPipelineRunWrite in data_classes library! that’s why I am unable to import it for my use. 

Userlevel 4

@Shweta what version of the python sdk are you using?  You many need to upgrade.
Try just using ExtractionPipelineRun




Yes @Jason Dressel  upgraded the version and used the above classes like this and I am getting below error!
creds = OAuthClientCredentials(







    cnf = ClientConfig(client_name=clientConfig.project, project=clientConfig.project_name, credentials=creds, base_url=clientConfig.base_url)

    client = CogniteClient(cnf)

class usage:
    newSuccessRun = client.extraction_pipelines.create(ExtractionPipelineRunWrite( extpipe_external_id="your_ext_id",status="success", message="Error message1"))


and same below error with ExtractionPipelineRun.


TypeError: 'extraction_pipeline' must be one of types [<class 'cognite.client.data_classes.extractionpipelines.ExtractionPipelineCore'>, typing.Sequence], not <class 'cognite.client.data_classes.extractionpipelines.ExtractionPipelineRunWrite'>

can you help me with this?

Userlevel 4

Sorry for the delay.  It appears there may have been a bug in the SDK that is now fixed.  Please try with the following code snippet:

run = ExtractionPipelineRunWrite(extpipe_external_id="csv:oee:datapoints", status="failure", message="custom error message")
result = client.extraction_pipelines.runs.create(run)



Yes, thank you very much for you help, issue is resolved.