Skip to main content
Solved

CDF Toolkit Instances deployment using csv file

Ayush Daruka
Seasoned

Can we use a CSV file instead of a Yaml file containing details of the instances we want to populate in a view using CDF toolkit? If yes, is there a limit to it? 
 

Edit:  What I mean to ask is whether we can use a csv file directly to populate a view in a data model, i.e. without using raw and transformations? Just like we can use a node.yaml file to directly deploy an instance in a view.

Best answer by Anders Albert

 ​@Ayush Daruka We have release Toolkit `v0.4.7` which has alpha support for populating nodes through a view from a csv or parquet file. 

You enable it in the `cdf.toml` with the following

[alpha_flags]
populate=true

The command is `cdf populate view`. 

View original
Did this topic help you find an answer to your question?

9 replies

Ayush Daruka
Seasoned
  • Ayush Daruka
  • Author
  • Seasoned
  • 24 replies
  • February 6, 2025

​@Snehal-Jagtap 

K
MVP
Forum|alt.badge.img+4

We have some of our instance data in csv files, and it works well. Our files are structured like this;

key name description aliases source ...
external id 1 name 1 ... ... ... ...
external id 2 name 2 ... ... ... ...

The files are deployed to raw via cognite toolkit and then instances are populated with transformations/workflows. 

None of our CSV-files are very large though. I think the largest ones are approximately 300 rows.

P
1 person likes this
  • P
    palronning
Ayush Daruka
Seasoned
  • Ayush Daruka
  • Author
  • Seasoned
  • 24 replies
  • February 6, 2025
Kristian Nymoen wrote:

We have some of our instance data in csv files, and it works well. Our files are structured like this;

key name description aliases source ...
external id 1 name 1 ... ... ... ...
external id 2 name 2 ... ... ... ...

The files are deployed to raw via cognite toolkit and then instances are populated with transformations/workflows. 

None of our CSV-files are very large though. I think the largest ones are approximately 300 rows.

Thanks for the information. Have you also tried using the csv directly to populate a view just like we can with a node.yaml file?

K
MVP
Forum|alt.badge.img+4

No, our node.yaml files only contain node types, no instances.

APSHANKAR Sagar
1 person likes this
  • APSHANKAR Sagar
    APSHANKAR Sagar
Anders Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Anders Albert
  • Seasoned Practitioner
  • 96 replies
  • February 11, 2025

​@Ayush Daruka as ​@Kristian Nymoen is saying only a YAML file is supported and it is intended for either node types or configuration nodes. For populating large amount of nodes transformations/or custom script/an extractor are the only options.

Note I have logged this as a feature request for Toolkit.

Ayush Daruka
1 person likes this
  • Ayush Daruka
    Ayush Daruka
Ayush Daruka
Seasoned
  • Ayush Daruka
  • Author
  • Seasoned
  • 24 replies
  • February 11, 2025
Anders Albert wrote:

​@Ayush Daruka as ​@Kristian Nymoen is saying only a YAML file is supported and it is intended for either node types or configuration nodes. For populating large amount of nodes transformations/or custom script/an extractor are the only options.

Note I have logged this as a feature request for Toolkit.

 

​@Anders Albert Thanks for confirming!!!

Anders Albert
Seasoned Practitioner
Forum|alt.badge.img
  • Anders Albert
  • Seasoned Practitioner
  • 96 replies
  • Answer
  • February 13, 2025

 ​@Ayush Daruka We have release Toolkit `v0.4.7` which has alpha support for populating nodes through a view from a csv or parquet file. 

You enable it in the `cdf.toml` with the following

[alpha_flags]
populate=true

The command is `cdf populate view`. 

Mithila Jayalath
Ayush Daruka
2 people like this
  • Mithila Jayalath
    Mithila Jayalath
  • Ayush Daruka
    Ayush Daruka
Ayush Daruka
Seasoned
  • Ayush Daruka
  • Author
  • Seasoned
  • 24 replies
  • February 13, 2025

​@Anders Albert Thanks for sharing this. Let me try it out and get back if I have any questions.

Mithila Jayalath
1 person likes this
  • Mithila Jayalath
    Mithila Jayalath
Ayush Daruka
Seasoned
  • Ayush Daruka
  • Author
  • Seasoned
  • 24 replies
  • February 13, 2025
Anders Albert wrote:

 ​@Ayush Daruka We have release Toolkit `v0.4.7` which has alpha support for populating nodes through a view from a csv or parquet file. 

You enable it in the `cdf.toml` with the following

[alpha_flags]
populate=true

The command is `cdf populate view`. 

​@Snehal-Jagtap ​@Neerajkumar Bhatewara 

Reply


Terms & ConditionsCookie Settings

Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings