Hello,
We are heavely relying on transformations to transform our data from Raw (staging) service to Data Models. We notice that it has very high latency compared to a “simple” Spark job. As per our discussion with our Solution Architect we understood that the bottleneck is the Raw service.
Any plans to improve this in the future?
Thank you!
Hi!
We have plans in many fronts directed to improved UX, reliability and stable performance, this includes (but isn’t limited to) work we’re doing on the UI, Transformations as well as in RAW and Data Model Storage services. These are contained in the Data Onboarding topic of our roadmap.
For details specific to your use case, a little more information or examples about the issues you’re facing would be really useful in diagnosing them and suggesting fixes or workarounds that may be currently available
+8
Hi
An example could be a simple COUNT(*) from a staging table having around 10M instances takes minutes to execute where its expected to take some seconds (compared to other SQL engines/ Spark clusters).
Thanks
+8
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
Check the
documentation
Ask the
Community
Take a look
at
Academy
Cognite
Status
Page
Contact
Cognite Support
