Skip to main content
Active ⭐️⭐️⭐️
Girish S
Active ⭐️⭐️⭐️
Apr 26, 2023 8:32 AM UTC
Solved

Overriding the existing transformed table's data with new data by using transformation layer.

I have been using RAW tables and creating a final table out. After creating a transformation, I could see the number of rows as ~8000 but that's a wrong number. After changing the transformation a bit, now I got the correct number post running the transformation which  is around 4000 but the final RAW table isn't changing.

I could see that the table isn't being overridden with the new data. Can anyone explain me why?

 

Thanks in advance!

    JaimeSilvaAnswer
    Practitioner ⭐️⭐️⭐️
    Apr 27, 2023 1:58 PM UTC

    Hi! some reasons why you could see this are:

    • you could be seeing a 4000 rows read metric, and no write metric. Maybe your query could be reading the expected amount of rows, but none is fulfilling the WHERE condition, so they are all discarded, if not a single row is written, the write metric is not present.
    • you could be writing to the wrong table. Double check your transformation destination, and see that it coincides with the table you are checking.
    • you could be writing to the wrong project. If you have access to more than one project, and you set your transformation credentials manually, it’s possible that you entered the wrong credentials, and are writing to another project than the one you are logged in on Fusion.

    If you provide us with some more details of your transformation (a sanitized version of the query, for example), we’ll be happy to help you further.

    View in thread

    3 replies

    Practitioner ⭐️⭐️⭐️
    JaimeSilvaAnswer
    Practitioner ⭐️⭐️⭐️
    Apr 27, 2023 1:58 PM UTC

    Hi! some reasons why you could see this are:

    • you could be seeing a 4000 rows read metric, and no write metric. Maybe your query could be reading the expected amount of rows, but none is fulfilling the WHERE condition, so they are all discarded, if not a single row is written, the write metric is not present.
    • you could be writing to the wrong table. Double check your transformation destination, and see that it coincides with the table you are checking.
    • you could be writing to the wrong project. If you have access to more than one project, and you set your transformation credentials manually, it’s possible that you entered the wrong credentials, and are writing to another project than the one you are logged in on Fusion.

    If you provide us with some more details of your transformation (a sanitized version of the query, for example), we’ll be happy to help you further.

    Dilini Fernando
    Expert ⭐️⭐️⭐️⭐️
    Dilini Fernando
    Expert ⭐️⭐️⭐️⭐️
    May 8, 2023 12:27 PM UTC

    Hi @Girish S,

    I hope Jaime’s reply was useful. If you are able to share more information about transformation, then we are able to further help you.

    Best regards,
    Dilini 

    Dilini Fernando
    Expert ⭐️⭐️⭐️⭐️
    Dilini Fernando
    Expert ⭐️⭐️⭐️⭐️
    May 17, 2023 6:18 AM UTC

    Hi @Girish S,

    As of now, I will close this thread. If you have any questions please feel free to post here.

    Best regards,
    Dilini 

    Terms & ConditionsAccessibility statement