Join the conversations to shape a safer, more efficient, and sustainable industrial future!
Recently active
In this conversation I will explain / help how to access the data from the Open Industrial Data Project (OID) CDF Project, through the Cognite Python SDK, with authentication through an interactive login token from Cognite Hub’s Azure Active Directory. The video in the article will step by step show you how you can get access to the data, and also link to the relevant information you will need. Steps:1. Install or update the following python modules/librariespip install cognite-sdkpip install msalpip install pandas2. Download the publicdata.py file and store it in your python environments working directory. this file is prefilled with the OIDC ID’s and CDF project values needed to access the CDF project.3. Open a Jupyter Notebook or python file and import the publicdata.pyfrom publicdata import c4. Access the OID CDF project through the CogniteClient object: c, and when the first method is called on the CogniteClient object it will authenticate and redirect towards the AAD tenant and u
We proudly announce that Flexible Data Modeling has reached public beta. Flexible data modelling in CDF enables you to model industrial data the way you understand it in a self-documented way so that domain experts can find, recognize and understand the data without a third-party manual.pdf. Your data is stored in one single knowledge graph, and empowers users to search, filter and aggregate data according to their needs through various data models on top of the knowledge graph. Flexible data modeling enables rapid scaling, whether it is building an app, data science model, dashboard, or other use cases, we hope that this capability can be a great toolkit in helping you to express data in the most intuitive way.In the Beta version you can Create and manage data models in Fusion and using a cli tool Ingest data into the data model using CDF transformations Query data using GraphQL with powerful search and filtering in Fusion or your own applications with even more powerful features
OpenID Connect has been enabled on Open Industrial Data for three months.We will be removing the option to use API keys for authentication on November 1st, 2022.If you are still using API keys for authentication, please change the Authenticatioon to openID Connect. Learn how to do that in this article.
Developing, tracking and meeting sustainability goals is becoming increasingly important for companies in the public sphere. Many of them are turning to data-driven solutions to help them monitor, report and reduce their environmental impact. Recent regulations in Europe recognises that Data-driven solutions for GHG emissions reductions are contributing to climate change mitigation (see official site from the European Commission - link) At Cognite, we have encountered many innovative solutions that promote sustainability. Examples include: Automating the recording and reporting of greenhouse gas emissions in industrial processes. Optimizing energy efficiency at the equipment and system level to minimize scope 2 emissions. Using robotics to detect dangerous leaks. Leveraging operational data to derive the environmental footprint of discrete products. Protecting biodiversity with the automatic detection of birds close to onshore windmills After analysing dozens of those solutio
Hello Charts Early Adopter Community,I know it’s been a bit quiet in this group lately, but rest assured, the team and I have been hard at work. We have just released several new features and fixes to production. You can scroll down to read about each of them in detail.Please do leave comments below with questions and feedback!What’s coming next?We’re in the middle of our development cycle in preparation for the next major release of Cognite Data Fusion in December 2022. There are the two major features our team is working on that you can look forward to having available:Moving Charts (charts.cogniteapp.com) into Cognite Data Fusion (fusion.cognite.com) We’ve received plenty of feedback that our product experience can and need to be more tightly integrated and we’ve heard you loud and clear. As you all know, Charts is currently available on charts.cogniteapp.com, which is separate from Cognite Data Fusion (fusion.cognite.com). Despite starting out on its standalone URL, Charts has alw
Do you know what courses are available for you to explore the data from the Open Industrial Data project?Cognite Academy creates e-learning courses with an emphasis on providing hands-on experiences so you can learn to use Cognite Data Fusion through interacting with real industrial data. In most of our courses, the training data we use comes from the Open Industrial Data (OID) project, a live stream of industrial data from the Valhall oil platform. While the data is from the oil and gas industry, it is relevant for all asset-heavy industries as it provides insights into dynamic industrial processes. In this article, I will share a series of CDF & Power BI courses where you can explore the Open Industrial Data and solve a simple use case in Power BI: Introduction to CDF & Power BI In this course, you will connect Power BI to CDF, and retrieve the Open Industrial Data to learn how filtering and aggregation work with the connector in Power BI. CDF & Power BI: Solving
Are you a Power BI user who wants to gain hands-on experience with Cognite Data Fusion? Are you curious to learn how to handle a large set of CDF data in Power BI?The Cognite Power BI connector lets you use a CDF project as a data source in Power BI Desktop to query, transform and visualize data, share insights across your organization, or embed dashboards in your app or website. When working with the Cognite Power BI connector, it is important to know how to refresh the data and apply best practices to deliver accurate results. The newest course from Cognite Academy introduces you to implementing incremental refresh - one of the best practices to get the most out of the Cognite Power BI connector. Our instructors first explain when to use incremental refresh, then show you how to set it up in four steps. In this course, the training data comes from the Open Industrial Data (OID) project, a live stream of industrial data from the Valhall oil platform.After completing this course, you’l
If you have lost access to your previous device, you can reach out to support@cognite.com and request to re-register for multi-factor authentication. You can also reach out to support here.
Morten Andreas Strøm / Ben Skal September 12, 2022 What makes Cognite unique? Why is partnering with Cognite the best investment of your time and resources?This is a 3 part series where @Morten and I answer these questions through an indepth look at how our product, Cognite Data Fusion, can help you use industrial data to ignite your digital roadmaps. The topics we discussing are: What is Cognite Data Fusion and why did we build it? (First post) Data modeling grounded in business impact (Previous post) The opportunity cost of custom building your industrial data platform (This post) In the first Why Cognite post, we discussed the data problem Cognite Data Fusion is built to address. The short answer, industrial companies need simple access to complex industrial data. The reason, most operations teams have many business opportunities, but are struggling to effectively use data to improve production. In the
This post is a hands-on introduction to the features supported in the Transformations Python SDK.Prerequisites Use Case 1: Triggering Transformations Step 1 - Create RAW Tables Step 2 - Uploading data to RAW using Postgres Gateway Step 3 - Create new SQL Transformations Step 4 - Trigger the transformation from Azure Data Factory Use Case 2: Orchestrating Transformations Step 1 - Create RAW Tables Step 2 - Create new SQL Transformations Step 3 - Orchestrate Transformations in sequence PrerequisitesKnowledge: Basic knowledge of Azure Functions and Azure Data Factory Basic knowledge of Cognite Data Fusion RAW and SQL Transformations Prior experience with Python, Postgres and SQL Required Datasets:Download and Unzip the attached hub.zip file, you should find the below structure Use Case 1 : asset-hierarchy.csv UseCase 2: OID-Asset-hirerachy.csv OID-Timeseries.csv OID-Datapoints.csv Use Case 1: Triggering TransformationsData is extracted from source systems and
What makes Cognite unique? Why is partnering with Cognite the best investment of your time and resources? This is a 3 part series where @bskal and I answer these questions through an in-depth look at how our product, Cognite Data Fusion, can help you use industrial data to ignite your digital roadmaps. These posts are for those of you who are new to using Cognite Data Fusion and want to understand how we approach the challenges of working with industrial data without losing focus on delivering business impact. The topics we discussing are: What is Cognite Data Fusion and why did we build it? (Last post) Data modeling grounded in business impact (This post) The opportunity cost of custom building your industrial data platform (DIY) In the first Why Cognite post, Ben and I shared why we built Cognite Data Fusion. The short answer, industrial companies need simple access to complex industrial data. The reason, most operations teams have many business opportunities, but are strugglin
Hi, I'm Damjan, and I work with research in the Cognite Data Onboarding group. We’re on a mission to improve and streamline the data onboarding experience for existing and future Cognite Data Fusion users.Our current focus is connecting data sources, building extraction pipelines and the needs around data onboarding. What's your challenges, needs and expectations with regards to the core Cognite Data Fusion onboarding experience? Shout out, share your thoughts and comments below. Thanks for helping us improve CDF!
Cognite Data Fusion is a product built to address the challenges of working with industrial data by: Making data available - Liberate their IT, OT, ET and visual data from siloed source systems with our extractor pipelines. This is done reliably and at scale. Making data meaningful - We use AI-powered contextualisation services to create an Industrial Knowledge Graph that delivers trusted, contextualised data Make data valuable - Cognite Data Fusion enables your teams to access this data with the best-of-breed tools of your choice to turn this data into business value Monolith solutions often end up creating vendor lock-in and can even end up creating more data silos within your organisation With trusted, contextualised data available in an industrial knowledge graph, your teams are equipped to scale solutions both in the volume of new solutions and replicating successful solutions across assets, lines, or sites.The videos are based on the “ice cream factory” use case: a use case
Morten Andreas Strøm / Ben Skal August 22, 2022 Hello digitalization community. My name is Ben Skal, and this is my first time posting to our community. At Cognite, I am part of our industry team and focus on helping our customers apply Cognite Data Fusion to solving the most difficult challenges within their operations. I’ve spent my career working in industry (11 years and counting). First, for a global steel company, then at a major process automation company, and now at Cognite. I am currently living in Austin, Texas and looking forward to e-meeting and learning from this community. The purpose of this series is to answer the following questions: What makes Cognite unique? Why is partnering with Cognite the best investment of your time and resources? This will be a 3 part series to precisely answer these questions through an in-depth look at how our product, Cognite Data Fusion, can help you use ind
In the 1800s, enterprises organised themselves to use their capital assets effectively. Beginning in the mid-1900s, they organised to take better advantage of their people. Today, “data” are increasingly important to virtually all companies. There are many ways to “put data to work,” each with its own strengths and challenges. One option is to focus on finding and exploiting both value pools for the business and deep, fundamental technical capabilities provided by CDF. This can be done by executing an onsite Use Case Discovery Workshop. There are three main steps to executing a Use Case Discovery Workshop: Identify qualified use case ideas Prioritize the use case ideas and select the top use cases Detail out the top use cases 1. Identify qualified use case ideas 2. Prioritize the use case ideas and select the top use cases 3. Detail out the top use cases How do you find the best opportunity to leverage data ?
We are very excited about being officially in General Availability with Cognite Functions! A big thank you to everyone who helped in this journey and your tremendous contribution! Please keep posting feedback and issues, as we are constantly improving the service.As part of GA, there are a few things that you should consider:We have support in the official SDK (cognite-sdk version 3.9.0) and have moved to V1 API endpoint. We recommend you to use only the official Python SDK when creating new functions and migrate the old functions that point to the experimental one. We will remove Functions from the cognite-sdk-experimental starting version 0.94.0. You will still be able to use the experimental SDK with versions < 0.94.0 until we remove the playground API (because the experimental SDK uses the playground URL) by November 1st. Functions in API playground is retired at 1st of November.Check out here more details about the release:
We’re @Uzair Wali and @kelvin, Senior Data Scientist and Data Science Lead in Cognite’s Manufacturing delivery team. In this post we talk about the increasingly important ability to intuitively and flexibly query data from all steps of a product life cycle and across source systems, with examples we’ve implemented on Cognite Data Fusion together with our users.The need for traceabilityIn many manufacturing industries, the ability to trace a product through its manufacturing life cycle, whether internal or supply chain is extremely important. It entails the collection and management of information regarding what has been done in manufacturing processes, from the raw materials and parts used to the shipment of finished products. An industrial knowledge graph that enables this traceability has the potential to not only let users speed up or automate existing use cases, it also opens up possibilities for considerable value addition.Typical questions A customer complains about the quality o
Hello Charts Community,Let me begin by saying thank you for all of your input, feedback, and contributions you’ve provided thus far. On behalf of the entire team, we couldn’t have made Charts into what it is today without your invaluable contributions. Charts General AvailabilityFor our August 2022 Cognite Data Fusion release, we have announced that Charts is transitioning from early adopter to general availability! We are eager and proud to move this valuable functionality into its next phase of life.In practice, it’s a stamp of approval that Charts is a reliable and stable Cognite Data Fusion feature. As we have done throughout our early adopter phase, we still intend to release new functionalities continuously and as soon as they’re ready to be made available. We will roll-up the communication in our bi-monthly CDF release communications, but will post in this group as soon as anything new is ready for use. What’s next for Charts?For the remainder of the year, our key focus area w
In the spirit of summer reading- here’s a pretty interesting blog post covering 5 emerging challenges in commodity trading. What similarities/differences/additional challenges apply to power trading? Anything missing here?https://www.cognite.com/en/blog/commodity-trading-data-challenges1. Rapid increases in the number and availability of new data sources are accelerating the complexities of managing data and analytics in global markets2. Reliance on legacy systems means participants in high-paced commodity markets struggle to make data relevant and actionable3. Participants in commodity markets need to accelerate their data management and digital development just to keep up with technological improvements4. Existing data solutions in commodity markets are often designed as one-stop single solutions or platforms, with little room to create a proprietary competitive edge5. Continued growth in market complexities will require that IT platforms and data architectures are designed to remain
Background: A production facility had numerous valves that were being opened and closed at varying rates and amounts. The subject matter expert at hand wanted to be alerted when the pressure values of these valves exceeded or dropped below a certain value as set by the subject matter expert.Problem: Today, the majority of valve maintenance is being done according to a fixed schedule. Without AIR and CDF the pressure sensors on these valves would have to be manually read and then converted into an excel spreadsheet. Once the data has been extracted into excel the subject matter expert had to analyze this data and figure out if the thresholds were being breached. If they were then the maintenance is done in order to prevent accidents. As it can be inferred this process was very manual and not easily scalable. The subject matter expert also wastes their time on tedious tasks as compared to actual important tasks.Solution: Using AIR a data scientist is able to easily define and deploy a t
Hello Charts Early Adopter Community,There have been several new features and functionalities released lately, with some being released just today. I’ve recorded a video (below) to explain several of these core features in detail, namely – calculations running on individual data points, not aggregates.You can also scroll down to read the written details.Please do leave comments below with questions and feedback!DetailsCharts UI/UX Calculations are now running on individual data points, not aggregates In the past, calculations have been running on time series aggregates. This meant that, although calculations were approximately calculations were often not correct his is a very important new feature that greatly increases the accuracy and trustworthiness of calculation results in Charts. Watch the above video or see the slides, below, for more info. If you receive a warning that looks like this (see image, below), then it means that the results of the data has been downsampled to perfo
WhyWhen starting out in Cognite Data Fusion (CDF) project, it's natural to start by creating data governance elements like groups, datasets, and Raw databases from the CDF user interface. But as the solution begins to scale, you'll quickly realize that it is demanding to set up a detailed configuration handling multiple solutions, sources, roles, and other dimensions. For scaling, precise control is needed for access management and data governance and to enforce the guidelines and rules across the solution. The problem is not trivial, and a good way to solve it is by replacing the manual approach with a configuration-driven system, where the configuration language supports higher-level concepts for data lineage and access control. With configuration files as the foundation, you can set up an automated DevOps process and review and approve any changes to the structure before they are deployed. This approach also dramatically simplifies sharing the same configuration across multiple envi
Just stumbled upon an article from McKinsey in 2019 which articulates well what I believe is a crucial role at many of the customers I’ve worked with: the Analytics Translator. The technology in industry is continuing to move forward quickly opening up many new opportunities, but the deep domain expertise in production processes, maintenance, and more is still as important as ever. To me, the Analytics Translator role seems a perfect bridge between these two worlds. I’ve seen a number of manufacturers successfully “upskill” prior machine operators, process engineers, and more to fulfill this role. It’s always very impactful, making for significantly more effective digital initiatives. From my personal experience, I see them doing things like:Running use case workshops and prioritizations Internal training of users Adoption tracking Interim project manager during new technology initiatives Liaison with external technology partners and SIs Curious how others see this role, and whether yo
Check the
documentation
Ask the
Community
Take a look
at
Academy
Cognite
Status
Page
Contact
Cognite Support
