Skip to main content
Implemented

Selecting a data model in transformations improvement

Related products:Transformations and RAW
  • April 19, 2024
  • 9 replies
  • 191 views
  • ibrahim.alsyed
  • rsiddha
  • Lucas Carvalho de Sousa
  • Lucas Leocardo Araujo
  • Rodrigo Nunes Falcão
  • Leandro de Lima Almeida
  • Marissa
  • Saulo Dias
  • Marcos Falcão y Martin
  • Guilherme Marques Ribeiro
  • Pedro Machado
  • Gustavo Henrique Tostes
  • Bruno Muniz Nicacio
  • sebastianheibo.akso
  • viniciusperes
  • Luciana Martins Arêas da Costa
  • Larissa Trindade
  • Bruno Arruda Martins
  • Lucas Fabricio Lopes
  • Marina Souza
  • Leandro Souza dos Santos
    Leandro Souza dos Santos
  • DerekMeese1

The current selection of the target data model when creating in a transformation is sort of a lazy load:

Only these three data models are available when dropping down. There are dozens of data models in the project

 

However, as the project grows, the loading of the data models can take too long, since it’s very possible to exist dozens of data models, and it loads the data models one by one. Also if you close the pop up it will start over.

 

I suspect this is because it’s fetching all the information about the data models - all versions and views available - to fill the other options in other fields. And that could be too much to load all at once.

 

The proposal is to fetch all data models at once, but only their names. And then once the data model is selected, fetch the rest of the information from that data model only. 

 

This way we can have a more responsive UI when selecting these and reducing the amount of unecessary data we are fetching.

9 replies

  • Seasoned Practitioner
  • 133 replies
  • April 22, 2024

Hi @Marcos Falcão y Martin,
 

We have received your request. Our PM, @Elka Sierra,  will review your proposal and come back to you.

Thank you,
Andreea.

 


Jørgen Lund
Seasoned Practitioner
  • Product Manager
  • 113 replies
  • May 14, 2024

Hi @Marcos Falcão y Martin, thanks for this great feedback! 

Just to confirm: you are currently experiencing slowness/bad performance in the project you’re working in? 

We’ll look into improving how the list of models is loaded.

We’re also considering improvements to the target selection flow for Transformations in general. If you happen to have more feedback, feel free to leave it here - all input is appreciated!


Forum|alt.badge.img

Hello @Jørgen Lund, hope you are doing well

 

Yes, this is a reccurring issue in all the projects I had to create a transformation for - I believe it’s an issue in the API call itself when the project has too many data models already


Jørgen Lund
Seasoned Practitioner
  • Product Manager
  • 113 replies
  • February 6, 2025

Hi ​@Marcos Falcão y Martin, apologies for taking some time to get back to you - could you check and see if you still experience the slow loading issue? We’ve made some changes and are not able to reproduce it on our side anymore, so just wanted to confirm if you’re seeing the same improvement


Forum|alt.badge.img

Hello ​@Jørgen Lund! No worries.

 

I’m not experiencing the lazy loading anymore i.e., all the data models load at once.

However, we now have a big initial loading that takes something like 2 minutes or more to run (celanese-dev project)

I’ve noticed in the console it does multiple calls to the “tables” endpoint - I think one for each data model in the project.

 


Jørgen Lund
Seasoned Practitioner
  • Product Manager
  • 113 replies
  • February 13, 2025

Hi ​@Marcos Falcão y Martin, thanks for the update. Could you provide some more context on what you describe above? When are you seeing 2 minute loading windows? Is it in the Transformations UI?


Forum|alt.badge.img

Hey ​@Jørgen Lund! Correct, this is the transformation UI, when selecting the target data model.

 


Jørgen Lund
Seasoned Practitioner
  • Product Manager
  • 113 replies
  • February 26, 2025

@Marcos Falcão y Martin - I’m not able to reproduce this, I see no calls to /tables. Could you share a full screenshot (or a few screenshots) for where in the UI flow you’re seeing these calls and slow loading so I’m sure we’re looking into the right areas? 


Forum|alt.badge.img

@Jørgen Lund 

No data model option available to select

Multiple calls to tables endpoint. While these calls are not finished, the data models don’t load


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings