Hello Community,
cdf stores the data in cloud, could anyone know what is the database / database structure being used,
and any reason for using Spark SQL for transformations.
Best answer by Dilini Fernando
View originalHello Community,
cdf stores the data in cloud, could anyone know what is the database / database structure being used,
and any reason for using Spark SQL for transformations.
Best answer by Dilini Fernando
View originalHi!
The short answer wrt using Spark SQL for the transformations is that we use Spark as a technology component in the Transformations Service/API.
As for the database or DB structure used in CDF, the answer depends on the service. We don’t use a single DB or DB schema, and in some cases we don’t even use a “default” DB schema at all (the “schema” - but it isn’t actually a schema - depends on the data).
Since we expect our users (developers) to - for those we have available - interact with the resources through the API surface, I'm unclear as to what you’re wanting to achieve by having the requested information?
Can you take a look at these two documents that covers Backup and Disaster recovery:
I think they may contain some of the information you are looking for if your question is about how we organize the storage backends in general.
Hi
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.