Skip to main content

How to test and validate in isolation in CDF?

  • May 29, 2026
  • 0 replies
  • 16 views

Soumaya ZAMMIT
Seasoned ⭐️
Forum|alt.badge.img+3

Hi,

We're the Drilling & Wells Data Analytics Platform team at TotalEnergies. We use CDF as the backbone for our drilling data for ingesting, transforming, and exposing data across multiple countries and data products. We manage our CDF resources with cognite-toolkit and deploy through GitHub Actions.

As our data model grows and our team scales, we keep running into the same question: how allow all our team members to safely test and validate a change before it hits production?
 

The tricky part is that CDF resources are shared. Two PRs deploying to the same test environment at the same time will overwrite each other's containers, views, and data, making it impossible to validate either change in isolation.

Our plan is to deploy a fully isolated copy of the data model for each open PR, then tear it down automatically when the PR close, orchestrated end-to-end by a GitHub Actions workflow. This is achievable because all our resources and space naming are handled in the config.<env>.yaml file. Github Action will create one with the dedicated configs for each opened PR before deployment.
On PR close, the workflow purges all instances from this PR specific space, then runs cdf-tk clean.

This gives each PR a self contained environment: schema, views, containers, and transformation output all land in one space and are cleaned up automatically with no manual intervention.

 

We'd love to hear your thoughts on this approach. Does it match challenges you've faced? Is there something simpler that works for your team?
 

More broadly, how are you approaching testing and validation in CDF especially  if you're using cognite-toolkit? Are you doing automated deploy-and-validate on PRs? Manual testing in a shared environment? Something else entirely?