Cognite Data Fusion Release: December 2021

The Product Release Spotlight Webinar recording for December can be found here. 

​Trusting the data in Cognite Data Fusion (CDF) is the core of managing, building, and using the Cognite product suite. Data engineers need sufficient insight into the staged data to take the next step on the ingestion journey and the new data profiling report on staged data in Cognite RAW caters for this. 

This CDF release also introduces the new Cognite Solutions Portal. The Cognite Solutions Portal collects applications, use cases, and other helpful links to solutions your organization has built on top of CDF. Users no longer have to keep track of many different URLs to access the information they need in their daily work.

Read more on this and all the other improvements for the CDF December release below:

Data profiling on staged data

To get in-depth knowledge about the data quality, discover patterns, outliers, and see other statistics in the new standard profiling report available on the Profile tab in Cognite RAW. Report your findings to the data owners and use this as a trigger to discuss topics such as the best fit for the primary key column and contextualization and to provide the best support for the end-users of the data. Keep iterating on the data integrations to improve the data quality and prepare the data transformation into the CDF data model. Read more here.

​

Get notified about interrupted data flow into CDF

Catch any issues that interrupt the data flow into CDF by setting up email notifications on the Extraction pipeline page. You can define the time condition for triggering the email. Learn how to set it up in this article. 

New tools for running SQL Transformations

New Transformations CLI

The new Transformations CLI replaces the old Jetfire CLI offering a better developer experience. Now, you can easily declare and manage variables in the configuration file without any hard-code. You install the CLI using the Python pip package manager. Read more here. 

APIs and Python SDK for Transformations

The Transformations APIs and Python SDK are promoted to Version 1. Using the APIs/SDK, data engineers can orchestrate transformations sequentially, making it more reliable and quick. Check out the  API docs and SDK docs for more information.

Access management for SQL Transformations

You can now manage access to SQL Transformations using the new transformations:read and transformations:write capabilities in your CDF group. 

We still support transformation access using the transformationsor jetfire groups for existing users to ensure backward compatibility. However, we recommend that you switch to managing access to Transformations using the new capabilities.

Introducing the Cognite Solutions Portal

As companies ramp up their digitalization and transformation efforts, the number of applications, dashboards, and use cases increases rapidly. As a result, end-users have to keep track of many different URLs to access the information they need. This is the problem Cognite Solutions Portal sets out to solve. It gathers all your CDF-enabled solutions in one place. 

Use the Cognite Solutions Portal to add applications, dashboards from analytics and visualization tools, and helpful links into shared spaces.

Explore data in 3D

The CDF Data explorer already helps solution builders and data scientists find, validate, and learn about the data in CDF. Get the full experience by navigating and searching for assets and viewing the details in 3D models on the new 3D tab in the CDF Data explorer.

Charts

Cognite Charts runs on top of  CDF and is a powerful tool for engineers and domain experts to explore, trend, and analyze industrial data. For more information, visit the Charts [Early Adopter] Group. 

This release has the following improvements: 

Automatic data alignment

Charts now automatically handle data alignment in your calculations. Time series data often have different sampling frequencies and time stamps, and it can be hard to do even the simplest calculations (e.g. timeseries_1 + timeseries_2), especially if you’re not proficient in Python.  Calculation results will be inaccurate and untrustworthy without properly accounting for data alignment.  

A new and improved no-code calculation builder experience

The no-code calculation builder has been redesigned and rebuilt on a new, powerful library.