Skip to main content
Gathering Interest

Transformation Schedule Heatmap

Related products:Transformations and RAW
Anita Hæhre
ane.eide
joar.saether
+13
  • Anita Hæhre
    Anita Hæhre
  • ane.eide
    ane.eide
  • joar.saether
    joar.saether
  • Stig Grafsrønningen
  • Pedersen Jon-Robert
  • ishita1983
    ishita1983
  • alfi
  • Kristian Nymoen
  • Thomas Røkke
  • Patrick Galler
  • Raluca Bala
  • Abram Ziegelaar
    Abram Ziegelaar
  • Lars Lie
  • Ole Thoresen
  • JON L
  • Giovanna Luna Rosseto
  • Andrew Wagner
    Andrew Wagner
  • KHAN MUDASSIR

Akash Sood
MVP

Hi Team,

Below is a requirement to increase user experience and performance while using transformations.

Current problem: As a data engineer, I do not have visibility of how many transformations are scheduled to run at a particular time period.

As a result, we just rely on manual check or intuition at what time we must schedule our transformations to not hit the concurrency limit or result in increased load on the cdf instance.

Proposed idea: A schedule heatmap between time of day against the count would help in this regard. The time should be divided into a group like 10-15 minutes to analyze how many transformations are scheduled within that time period. The count should also consider the repetitions based on the cron expression passed.

Note: This can be extended to functions and data workflows as well

@Aditya Kotiyal ​@Jørgen Lund 

2 replies

kelvin
Seasoned Practitioner
  • Seasoned Practitioner
  • 53 replies
  • February 28, 2025

Hi ​@Akash Sood! That’s an interesting product idea! Your goal is primarily to avoid concurrency/load issues, right?

 

I put together a dashboard like this with Streamlit in Fusion a while ago. Cleaned up the code and shared it here in case you want to take a look and see if this can help you in the short term. Should be able to run it in Streamlit in Fusion, just make sure to add the plotly dependency too :) 


Akash Sood
MVP
Forum|alt.badge.img+2
  • Author
  • MVP
  • 80 replies
  • March 4, 2025

Thanks ​@kelvin. Will try this out


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings