We're pleased to announce that SQL transformations (formerly known as Jetfire) is now available as an integrated part of Cognite Console.
Directly from your browser, use Spark SQL queries to transform data from the CDF staging area, RAW, into the CDF data model where the data can be queried by users and applications.
To get started:
- Check out our in-depth article to learn more.
- Navigate to fusion.cognite.com and select Transformations.
Create, schedule, and share Spark SQL transformations
Members of the transformations
group can use SQL transformations to:
- Write Spark SQL queries to select the data to transform and specify how it should be transformed.
- Preview the query results.
- Schedule transformations to run at regular intervals.
- Share transformations with other users in your project to allow them to edit, run, and schedule the transformation.
Write efficient queries
SQL transformations run on the latest version of the Cognite Spark Data Source and support Cognite API v1. You can:
- Efficiently query data from RAW tables and CDF resource types.
- Load data incrementally to transform only the data that has changed since the last transformation job ran.
- Use custom SQL functions, in addition to the built-in Spark SQL functions, to write efficient transformations.