Skip to main content
  • 207 Product updates

Cognite Data Fusion - October 2020 updates

As the leaves turn orange and red and autumn settles in Norway, we continue our rapid iterations to bring you new enhancements. We are introducing a huge enabler to further enrich your industrial data in Cognite Data Fusion (CDF): Relationships - now available in beta. We continue our investment to enable users to explore data using domain terms by adding support for labels on the Files resource type. Additionally, in a new dedicated directory property, you can now store information about the source system directory where the file originates from. These new features allow data engineers to model data in ways that enable a broad range of solutions. For all application developers and data scientists out there, we’re adding support for configuring alignment for synthetic time series. Our next major release, with a bundle of exciting features, is planned for December 2nd. We’re already doing exciting prototyping and beta testing around data discovery for solution developers, scheduling of data pipelines, and administration and monitoring of ingestion pipelines. Much to look forward to! More flexible modeling of industrial data through RelationshipsWith the introduction of the Relationships resource type, we enable far more powerful ways to model industrial data. While previously, you could only model equipment relations as a tree, you can now model data flexibly in a graph.In this release, we are moving relationships from Playground to Beta.Data engineers can use relationships to organize your assets into multiple structures, tailored for your organization's purposes. For example, you can organize assets by the process flow or the physical connectivity in a grid structure to let users efficiently locate data for nearby equipment.And while some users in your organization find it useful and efficient to organize the equipment by its function in a maintenance hierarchy, others may benefit from having them organized by physical location in a location-based hierarchy:In short, relationships give you the flexibility to organize assets in structures that are meaningful for your organization. See Modeling asset structures with relationships for more example structures.Beyond structuring assets, you can use relationships to create richer connections between data elements and build a knowledge graph on top of the data you already have.To learn more about how relationships can represent industrial reality, we recommend that you read the "Building a contextualized power system network model" whitepaper. Adding labels support for the Files resource typeWe continue our investment to enable users to explore data using domain terms. Labels are now supported on Files resource, available through the API, the Python SDK and the JavaScript SDK.from cognite.client import CogniteClientfrom cognite.client.data_classes import LabelFilterc = CogniteClient()my_label_filter = LabelFilter(contains_all=["WELL LOG", "VERIFIED"])file_list = c.files.list(labels=my_label_filter) As a data engineer, you can now classify files by what kind of file it is. Application developers can build good end-user experiences and let users browse files by category. For example, end users can browse files grouped as P&IDs, logic diagrams, and datasheets.Enhanced lineage information for filesThe new directoryPrefix property allows you to store and filter information about the directory in the source system where the file originates from. The new functionality is available in the API and the Python SDK.from cognite.client import CogniteClientc = CogniteClient()file_list = c.files.list( limit=5, directory_prefix="/usr/local/share/jupyter/") Configurable aggregation alignmentWe are introducing a new configuration option for aggregation in synthetic time series: alignment.New option enables you to:Create weekly aggregates that start on specific days of the week, for example, Saturday, or Monday. Create aggregates that align to different time zones.For example, the chart below shows the daily average chlorine indicator for city water aligned to two different time zones: GMT+0 and GMT+3.In the above graph, the new alignment option allowed us to align daily averages to timestamps in the desired time zones:Daily averages aligned to UTC/GMT+0: ts{externalId='houston.ro.REMOTE_AI[34]', alignment=1599609600000, aggregate='average', granularity='24h'} Daily averages aligned to UTC/GMT+3: ts{externalId='houston.ro.REMOTE_AI[34]', alignment=1599620400000, aggregate='average', granularity='24h'}By default, aggregation with synthetic time series is aligned to Thursday 00:00:00 UTC on 1 January 1970.The new parameter is available in the Synthetic Time Series API, and can be used as part of query expressions without needing to update your SDK, as well as part of custom queries in Cognite Data Source for Grafana 2.0+.

Related products:Product Releases

Cognite Data Fusion - August 2020 updates

The Cognite Power BI Connector New function: Time series aggregatesThe time series aggregation now accepts a list of IDs instead of a comma separated string. You can now use a column from another query directly as input to the function, making it much easier to use.Setting the CDF environment for a custom clusterIf a CDF project is on a custom cluster, you can set the CDF Environment to the URL of the API server for the cluster when you connect to CDF.You can also set a specific API version by appending it to the URL, for example v1. We also support shorthand notations. For example, api/v1 uses the default API endpoint and v1 of the API.Examples of supported CDF environment values:(blank) - uses the default setting (https://api.cognitedata.com) api - the same as https://api.cognitedata.com api/v1 - the same as https://api.cognitedata.com/v1 https://api.cognitedata.com https://api.cognitedata.com/v1 xyz - a custom cluster. The same as https://xyz.cognitedata.com xyz/v1 https://xyz.cognitedata.com https://xyz.cognitedata.com/v1Learn more about the Cognite Power BI connector. New SDK section in the developer portalThe new section gathers the most useful resources and links related to the CDF SDKs:Getting started with JavaScript SDK and Python SDK. Quick links to reference documentation and GitHub repositories. Additional guides such as authentication, upgrade guides for JavaScript SDKs. Release notes for SDKs are now part of the developer portal.Python SDK 2.1The new release introduces an improved interface to make it easier to work with labels.To learn more about the changes in new release, see the release notes. JavaScript SDK 3.0New major release separates SDK into 3 different packages:A core package that contains the SDKs infrastructure. This is used by the other packages and allows users to easily create their own SDK by just defining the interface. The stable package contains all the stable API endpoints. The beta package extends the stable SDK with beta API functionality.In addition to structural changes, the new version of the SDK introduces these changes:unit attribute is now exposed in data points objects. Fields that can have arbitrary string names (Raw, Metadata) are no longer converted to Date objects if their names are: createdTime, lastUpdatedTime, uploadedTime, deletedTime, timestamp, sourceCreatedTime or sourceModifiedTime.To learn more about the new release, see the release notes.TIPTo upgrade to the newest version, follow the migration guidelines.

Related products:Product Releases

Cognite Data Fusion - July 2020 update

June has been an exciting month in Cognite with several large improvements and quality-of-life enhancements. New and improved Cognite Data Source for GrafanaGrafana is a flexible and powerful dashboarding tool, where anyone can easily create any desired dashboard for real-time data.We’re launching a new Cognite Data Source for Grafana that runs on our API v1, so that you can take advantage of all the latest product features in your dashboards. Read more about the latest changes in our blog post or jump straight to the documentation to learn how to create awesome dashboards on Fusion.Synthetic time seriesAs a developer, you know that end users typically care about combinations of different time series. We’re now releasing synthetic time series, so that you can let Cognite Data Fusion handle some of the time series wrangling for you and easily combine time series data, constants, and operators to produce the data that you need. Learn more in our documentation.Explore data using domain termsWe've made it even easier for data consumers to work with data, by letting you explore data using your familiar domain language. IT and data engineers provide this improved experience for data consumers by defining, managing, and applying labels to data. Learn more in our blog post.Additional quality-of-life enhancementsCheck which types and subtypes of events are available, and how many events you have of each type, in a simple API call. See our event aggregation section in the API docs.Extending support for CDF API 0.5We are extending support for API 0.5 until 30th September 2020, in order to provide dashboard creators using Cognite Data Source for Grafana (1.x) with additional time to upgrade to the new version of the connector.Up-to-date information about supported versions and deprecations is available at API versioning.

Related products:Product Releases

Cognite InField

Operation Support becomes InFieldOur goal is to consistently improve Cognite products and services based on our customer needs. The Operation Support application has evolved and now targets more infield activities such as operational maintenance, and efficient preparation of tasks. Operation Support is therefore renamed to InField.InField is still as accessible as it has been and now also has a distinguished product name.Add images to checklistsYou can now add images to checklist items. This feature is available on the desktop and on the mobile app. To add an image, click the More button to open the menu for checklist items and then click Add image. Take photos by using your phone's camera and attach the photos to your checklists. You can optionally add a description to the photos.Operation Support endrer navn til InFieldVi jobber kontinuerlig med å forbedre Cognite sine produkter og tjenester basert på kundens behov. Operation Support har utviklet seg og retter seg nå inn mot flere felt-aktiviteter som vedlikehold og planlegging av oppgaver. Operation Support har derfor endret navn til InField.InField er forsatt like tilgjengelig som før, men nå med et tilpasset produktnavn.Bilder i sjekklisterDu kan nå legge til bilder i sjekklister via PC og mobile enheter. Legg til et bilde ved å klikke på knappen More for å åpne menyen for sjekklister og klikk deretter Add image. Ta et bilde med kameraet på telefonen og legg til bildet i sjekklisten. Du kan også legge til en beskrivelse av bildet (valgfritt).

Related products:InField

Cognite Data Fusion - Introducing live reusable data quality metrics

After we released the first version of time series data quality monitoring, we've seen an unprecedented demand for ensuring that time series data is continuously reliable.It's important for end users of apps and dashboards to know when they can -- and importantly when they cannot -- rely on data to make operational decisions. That means that you as a data scientist or application developer need to communicate to the end user what the data quality status is.You can now easily report live data quality status in dashboards and applications using our data quality monitoring service. The monitoring service creates live data quality metrics, available as time series in Cognite Data Fusion, that you can display in your application -- ensuring that end users know the quality of the data that they are using.Try it out! Learn more in our guide to report data quality status in apps and dashboards.To enable this in your project the service account used by the data quality monitoring service needs to be updated. Go to quality monitoring in Console and follow the instructions on the banner. Note: You need IAM capabilities to do thisBONUS UPDATE: Additionally, you can now efficiently troubleshoot data quality alerts in the app through our updated UI. Quickly see which time series break which rules, explore the historical time series data, and click through to find more details about the data set that the time series belongs to.

Related products:Data Quality

Cognite Data Fusion - April 2020 update

Data sets available in Cognite ConsoleData sets let you document and track data lineage, ensure data integrity, and allow 3rd parties to write their insights securely back to your Cognite Data Fusion (CDF) project.Data sets group and track data by its source. For example, a data set can contain all work orders originating from SAP, or the output data from a 3rd party partner's machine learning model. Typically, organizations have one data set for each of its data ingestion pipelines in CDF.To get started with data sets:Check out our in-depth article to learn more. Navigate to fusion.cognite.com, select Data sets, and follow the steps in our guide to create your first data set.Cognite's Power BI Connector certified by MicrosoftWe're excited to announce that Microsoft has certified the Cognite Power BI Connector and that the connector is now publicly available. The Cognite Power BI Connector enables data consumers to easily read, analyze, and present data from Cognite Data Fusion (CDF) in compelling visualizations in both Power BI Desktop and the Power BI service.You can find the connector in the Other category of the Get Data dialog in Power BI. Learn more in our in-depth article.New upcoming transformation capabilitiesWe're continuing our work to improve the asset hierarchy functionality in SQL Transformations. You will soon be able to ingest multiple asset hierarchies at the same time and also ingest asset sub-trees to any level of the hierarchy. The asset hierarchies and sub-trees will merge perfectly with your existing or newly built asset hierarchy.To give you a more precise and manageable overview of your jobs, we are also updating the UI with additional details about each job: status, time of schedule, last run, duration, and the option to resume or stop a schedule.

Related products:Product Releases