Overriding the existing transformed table's data with new data by using transformation layer.

Question

I have been using RAW tables and creating a final table out. After creating a transformation, I could see the number of rows as ~8000 but that's a wrong number. After changing the transformation a bit, now I got the correct number post running the transformation which is around 4000 but the final RAW table isn't changing.

I could see that the table isn't being overridden with the new data. Can anyone explain me why?

Thanks in advance!

icon

Best answer by JaimeSilva 27 April 2023, 15:58

View original

JaimeSilva · Accepted Answer

Hi! some reasons why you could see this are:

you could be seeing a 4000 rows read metric, and no write metric. Maybe your query could be reading the expected amount of rows, but none is fulfilling the WHERE condition, so they are all discarded, if not a single row is written, the write metric is not present.
you could be writing to the wrong table. Double check your transformation destination, and see that it coincides with the table you are checking.
you could be writing to the wrong project. If you have access to more than one project, and you set your transformation credentials manually, it’s possible that you entered the wrong credentials, and are writing to another project than the one you are logged in on Fusion.

If you provide us with some more details of your transformation (a sanitized version of the query, for example), we’ll be happy to help you further.

Dilini Fernando · Answer

Hi @Girish S,I hope Jaime’s reply was useful. If you are able to share more information about transformation, then we are able to further help you.Best regards,Dilini

Overriding the existing transformed table's data with new data by using transformation layer.

3 replies

Reply

Reply

Sign up

Log in to the community

Scanning file for viruses.

This file cannot be downloaded