Solved

Regarding cdf's DataBase and usage of Spark SQL

  • 25 October 2023
  • 4 replies
  • 41 views
V
Rank
Userlevel 1
Badge +1

Hello Community,

cdf stores the data in cloud, could  anyone know what is the database / database structure being used,
and any reason for using Spark SQL for transformations.

icon

Best answer by Dilini Fernando 9 November 2023, 08:43

View original

4 replies

T
Rank
Userlevel 3

Hi!

The short answer wrt using Spark SQL for the transformations is that we use Spark as a technology component in the Transformations Service/API.

As for the database or DB structure used in CDF, the answer depends on the service. We don’t use a single DB or DB schema, and in some cases we don’t even use a “default” DB schema at all (the “schema” - but it isn’t actually a schema - depends on the data).

Since we expect our users (developers) to - for those we have available - interact with the resources through the API surface, I'm unclear as to what you’re wanting to achieve by having the requested information?

joar.saether
Rank

Can you take a look at these two documents that covers Backup and Disaster recovery:

I think they may contain some of the information you are looking for if your question is about how we organize the storage backends in general.

C
Rank
Userlevel 3

Hi @Viswanadha Sai Akhil Pujyam, did the above help you?

Dilini Fernando
Rank
Userlevel 4
Badge +2

Hi @Viswanadha Sai Akhil Pujyam,

I hope the above helped. I’m closing this topic now, please feel free to create a new post if you still experience problems.

Best regards,
Dilini 

Reply


Terms & ConditionsCookie Settings