Skip to main content
Gaetan  Helness
MVP ⭐️⭐️⭐️⭐️⭐️
MVP ⭐️⭐️⭐️⭐️⭐️
March 1, 2023
Gathering Interest

Transformation data quality

Related products:Data Staging (RAW)
  • March 1, 2023
  • 1 reply
  • 27 views

Transformations are used quite extensively by Data Engineers. I feel more confident about my Python code as I can unit test. Then, if I make changes to my scripts, I have more guarantees that it will not cause changes in the data. There is no way to easily see the results of a code change in Transformations without having some other script/Transformation to investigate the changed data. This can result in many data bugs, especially if you change code that was written by someone else. It would be helpful to explore a functionality towards enabling the data quality of the transformations we write every day.

1 reply

Anita Hæhre
Expert ⭐️⭐️⭐️⭐️
Head of Community
March 3, 2023
NewGathering Interest
Anita Hæhre