CDF - Introducing SQL transformations

Related products: Transformations and RAW

We're pleased to announce that SQL transformations (formerly known as Jetfire) is now available as an integrated part of Cognite Console.

 

Directly from your browser, use Spark SQL queries to transform data from the CDF staging area, RAW, into the CDF data model where the data can be queried by users and applications.

To get started:

  1. Check out our in-depth article to learn more.
  2. Navigate to fusion.cognite.com and select Transformations.

Menu

Create, schedule, and share Spark SQL transformations

Members of the transformations group can use SQL transformations to:

  • Write Spark SQL queries to select the data to transform and specify how it should be transformed.
  • Preview the query results.
  • Schedule transformations to run at regular intervals.
  • Share transformations with other users in your project to allow them to edit, run, and schedule the transformation.

Write efficient queries

SQL transformations run on the latest version of the Cognite Spark Data Source and support Cognite API v1. You can:

  • Efficiently query data from RAW tables and CDF resource types.
  • Load data incrementally to transform only the data that has changed since the last transformation job ran.
  • Use custom SQL functions, in addition to the built-in Spark SQL functions, to write efficient transformations.
Be the first to reply!