Issues and limitations when using the CDF data upload plugin

I’ve started working with the data upload plugin for the CDF Toolkit, and I’ve run into a few issues that made the process a bit more difficult than expected.

1. Data plugin is not part of the build phase

I’m using the data plugin to seed some data. It’s essentially a one‑off upload that isn’t worth creating a full-facing component for, but I still want it to happen automatically as part of the deployment rather than manually.

When uploading data model instances using Manifest.yaml, the version field is required under the source dictionary. However, I don’t want this version to be a hardcoded value. I tried defining the version in config.{env}.yaml and expected the placeholder to be replaced during the build. This didn’t work, so I ended up doing the replacement manually in the CD pipeline.

2. Folders are not read recursively

After setting the correct version, the next step is uploading the files. I organized them across multiple folders, but only the first folder level was processed by the plugin. To work around this, I wrote a script that flattens the folder structure so all files can be uploaded.

There are cases where multiple files need to be uploaded, and keeping them organized in the repository would be much nicer than flattening everything manually.

Summary

I managed to make everything work with some extra scripting, but these features would be great to have built into the product.

Is this the expected behavior? Are there any plans or best practices to improve this experience?

Best,
Alex