Diana Chimnaz Johan

Seasoned ⭐️⭐️⭐️

Solved

csv_extractor extraction pipeline

2 replies
79 views

Hello,

I am running a csv_extractor that loads data to Raw explorer. I run the extractor from my machine locally through cmd. How do I set up an extraction pipeline to monitor this extractor? I created an extraction pipeline from “monitor extractor”, but not sure how to link it to my extractor.

Regards,

Diana

Gaetan HelnessAnswer

MVP ⭐️⭐️⭐️⭐️⭐️

Hi,

You can use the API https://api-docs.cognite.com/20230101/tag/Extraction-Pipelines-Runs/operation/createRuns or the SDK for that purpose https://cognite-sdk-python.readthedocs-hosted.com/en/latest/data_ingestion.html#cognite.client._api.extractionpipelines.ExtractionPipelineRunsAPI.create

If using our python extractor-utils library, you get that out of the box

https://cognite-extractor-utils.readthedocs-hosted.com/en/latest/uploader.html#extraction-pipeline-runs

Hope that helps, let me know if any additional questions

Gaetan HelnessAnswer

MVP ⭐️⭐️⭐️⭐️⭐️

Hi,

You can use the API https://api-docs.cognite.com/20230101/tag/Extraction-Pipelines-Runs/operation/createRuns or the SDK for that purpose https://cognite-sdk-python.readthedocs-hosted.com/en/latest/data_ingestion.html#cognite.client._api.extractionpipelines.ExtractionPipelineRunsAPI.create

If using our python extractor-utils library, you get that out of the box

https://cognite-extractor-utils.readthedocs-hosted.com/en/latest/uploader.html#extraction-pipeline-runs

Hope that helps, let me know if any additional questions

Diana Chimnaz JohanAuthor

Seasoned ⭐️⭐️⭐️

Thank you, this was very helpful!