Spark upgrade to 3.4 version

Hi ​@Marwen TALEB!

I can confirm that upgrading to Spark 3.4 is already on our radar. We recognize the value of this upgrade both for new features support, but also for security improvements. That being said, this upgrade involves some complexity due to changes in configuration and we need to ensure backward compatibility, so this is not a trivial upgrade for us. Based on our current planning, we're aiming to complete this upgrade sometime in 2025, with a target of having it resolved before the end of the year.

We'll make sure to update this thread once we have more concrete timeline information.

Hi ​@Everton Colling , any news on this please? Thanks

​@navin.r ​@Jørgen Lund could you check with the Spark team if an upgrade is in the works and revert back here?

​@navin.r  ​@Jørgen Lund any updates on this?