Streamlining Access Management in CDF with Group Membership

Hello!ūüėĄ¬†We're pleased to announce the introduction of group membership within groups in Cognite Data Fusion (CDF). This enables admins to add users directly to groups within CDF, simplifying the process of granting access and understanding who the members of a group are by checking it within CDF. It also significantly reduces¬†the burden on the customer's IT team by minimizing (or eliminating) the number of tickets/requests they receive to create groups in their IdP, add users to these groups¬†and then link the IdP groups to groups in CDF.After adding the required capabilities to a group in CDF, admins can now choose how to manage the group's membership -¬†either internally within CDF or continue to manage it via the customer‚Äôs IdP, as before.¬†Group membership managed within CDF (New)For group membership managed internally within CDF, there are two options: admins can grant access to capabilities in the group to all users in the organization or grant access to users by explicitly adding them to the group in CDF.1. Granting access by adding user to a group¬†‚Äč‚Äč‚Äč‚Äč‚Äč‚ÄčCDF admins click on "Create" or "Edit" a group in CDF Adds or modifies all required capabilities and scopes Selects List of users¬†and¬†then adds users as members to the group.¬†NOTECurrently, only users can be added as members to a group in CDF. Support for service account creation within CDF and adding them as members of a group will be available in the June releaseCDF admins cannot add a user to a group in a CDF project if the user has never logged in to the organization, as user profiles are only created upon successful login to an organization¬†2. Granting access to all users¬†in the organizationCDF admins creates a group in CDF Adds all required capabilities with scopes Selects¬†All user accounts¬†option under members.CAUTIONUsers can access a CDF project within an organization if they are a member of at least one group in the project. Creating a group with 'All user accounts' in a CDF project grants access to the project, with the capabilities mentioned in the group, to all users in the organization¬†Groups with "All user accounts" as members are prioritized at the top of the list in the group management UI to prevent any unintended access. We recommend creating groups with 'All user accounts' as members containing basic capabilities, such as view only (all read capabilities), to ensure a smoother first-time login experience for new users.

Related products:Authentication and Access Management

Search, Data Exploration and Industrial Canvas ‚Ēā New Features

Hi everyone! ūüĎčToday we released a range of new functionality, improvements, and bug fixes in both Industrial data search, Data Explorer and Industrial Canvas. Thank you to all who have provided feedback and posted suggestions to us here on Hub - many of you will recognize your feedback incorporated into what gets released today.¬†¬†Industrial data searchOur focus since the March release¬†has been around incremental improvements, fine-tuning some interactions, tidying up the user experience, and in parallel, working on some bigger customization functionality that we will release at the end of Q2.¬†For today, we want to share with you this;¬† We have added¬†sorting¬†of search results¬†to make it easier and quicker to find relevant data. You can now sort data (ascending/descending) throughout the user interface. The list of categories can be sorted on¬†name¬†or¬†count¬†it contains The list of search results can be sorted on all¬†properties¬†of the category Left: Sorting on Categories. Right: Sorting on Properties. The list of categories has gotten a face-lift with a¬†default max number of categories shown, and a ‚Äúshow more‚ÄĚ option if the number of categories is high.¬† The¬†filters and search queries added as part of your exploration journey are now preserved during your user session¬†to ease the exploration experience. When you start from the list of search results, you can now add filters and/or a¬†search to filter down the list of result, dig into the filtered data, and be able to¬†return to the list of filtered search without the need for going through the filtering steps again.¬† A new¬†Statistics¬†functionality has been added, enabling you to swiftly create visualizations of your data. This feature lets you categorize data based on specific properties and visualize aggregated metrics such as counts and averages. Please note, however, that the Statistics functionality is not supported for all data categories - you can use this feature for the categories listed under Other categories in the left hand sidebar.¬† The¬†Statistics¬†functionality can be used to create visualizations of you data.Our¬†AI Copilot has been further integrated into the search functionality, streamlining the process of identifying categories and creating filters through conversational language. When you know chose to use the Copilot, you will the results appearing in the same interface as when you perform regular search. Additionally, the Copilot is equipped to autogenerate visualizations of statistics, further enriching your data exploration experience. You still trigger the Copilot the same way as before, choosing ‚ÄúAsk Copilot..‚ÄĚ.¬†AI Copilot is now more integrated into the Search experience.Some smaller bug fixes include increasing the portion of text we show (like for descriptions), and showing the label¬†name¬†instead of the label¬†ID¬†(thank you for reporting this!)¬†Industrial CanvasToday we have released an improvement to the experience for finding and adding data to a canvas, by unifying the experience with the Industrial data search. When clicking ‚ÄúAdd data‚ÄĚ you‚Äôll now recognize the experience from Industrial data search, which gives a more visual way of browsing your data, before adding it to the canvas.You recognize how the experience of finding and adding data to your Canvas is unified with the Industrial data search experience.Data ExplorerThere are two main improvements released today, both a result of feedback received from you here on Hub.¬† To ease and speed up data exploration, we are now¬†preserving the applied filter(s) and search query while you browse in the details mode¬†(they are reset when you exit the details mode). When you apply filter(s) and/or Search query in details mode, these are preserved while you are exploring data ¬†We have made some¬†adjustments to¬†filtering logic¬†in the common filters. In the Metadata filter, you‚Äôll find that there are several keys, each typically having several values each. If you¬†select several values for the same key, these are treated with the OR operation, while between keys we now apply the AND operation. Examples:¬† In the Metadata filter, you select the key ‚Äúnetwork level‚ÄĚ, and two values. This selection is treated as: network level = production_subsystem¬†OR production_system.¬† If you select another key in addition, this selection is treated with the AND operation: network level = production_subsystem¬†OR production_system AND product_type = Gross. Combining the Metadata filter with other common or resource-specific filters is treated with the AND operation. Here we have applied the filters: network level = production_subsystem¬†OR production_system AND product_type = Gross¬†Let us know if you have questions or feedback!Best,¬†Sofie Berge, on behalf of Cognite Product Management

Related products:Product ReleasesIndustrial CanvasSearch and Data Exploration

Charts Updates and New Features

Hei Everyone! We're excited to announce the latest updates to Charts, designed to enhance your data visualization and analysis experience. In this release, we've focused on improving data viewing capabilities, introducing new features for better visual analysis, and enhancing monitoring job setup and validation. These enhancements are aimed at providing users with greater flexibility, control, and confidence in leveraging Charts for their data-driven decisions. a) Improved Chart Data Viewing and Trust in the data viewed - Increased the maximum number of raw data points viewable in a chart from 500 to 100,000.- Aggregate min/max shading is now enabled by default when switching to aggregate data. This substantial increase in data points allows for a more comprehensive analysis of data trends in RAW mode without the need to switch to aggregated views prematurely. This min / max shading being on by default should ensure that users are better aware of when they are viewing aggregated data. We hope that these changes should help build better trust in Charts and make the data look more similar to what is viewed in PI systems. b) Slider/Marker Functionality in Charts: - Users can set and delete multiple sliders/markers to compare values across various points in a time series. With this new functionality, users gain the ability to freeze values at specific points in time within a given chart. By setting multiple sliders/markers, users can effectively compare data values across different time series, enabling deeper insights into data trends and patterns. c) Backfilling of Monitoring Jobs - Receive a count of alerts that would have been triggered within the specified time frame when setting up a monitoring job   With this latest enhancement, users can now conduct historical runs of monitoring jobs before finalizing their configurations. Upon setting up a monitoring job, users will receive valuable insights into the number of alerts that would have been triggered within the specified time frame based on the configured thresholds. This functionality not only allows users to verify the effectiveness of their threshold values but also helps prevent the potential spamming of alerts by ensuring that thresholds are set at reasonable levels.  

Related products:Charts

Exciting Updates: Enhanced Data Integration in Cognite Data Fusion (CDF)

Greetings, everyone!We're thrilled to share some exciting updates regarding our ongoing efforts to enhance data integration within CDF (Cognite Data Fusion). These improvements are aimed at streamlining your experience and empowering you with more capabilities. Let's dive right in!New Features for the File Extractor:  1. OpenText Documentum REST Services Support:We've expanded the capabilities of the Cognite File Extractor to now seamlessly integrate with OpenText Documentum REST services, so that you can connect to multiple sources by only configuring one extractor. Check out the documentation for configuration details here.2. Advanced Filtering:Enjoy enhanced client-side filtering options based on metadata, allowing for precise filtration and complex rules tailored to your needs.3. Improved SharePoint Integration:Say hello to improved SharePoint support! We now offer support for multiple SharePoint sites, dynamic site discovery, and the flexibility to configure sites via URL. Plus, we've added the capability to detect deleted files, ensuring your data remains up-to-date.4. Enhanced File Handling:Say goodbye to size limitations! We now support files larger than 5 GB, facilitating seamless handling of larger files. Updates and Changes: 1. Continuous Background Uploads:File uploads now occur continuously in the background and in parallel, resulting in significantly faster speeds. Additionally, file traversal on sources has been optimized for a smoother experience.2. Revised Configuration Schema:We've refined the configuration schema for FTP and SFTP, aligning it with other file providers for consistency. Refer to our documentation for detailed parameter names.3. Improved Retries and Handling:Experience more robust retries and improved handling of interrupt signals, minimizing disruptions during operations.4. External IDs Enhancement:We've revamped the external IDs for local and Samba files, now based on file paths instead of underlying inodes. This ensures a seamless transition if you're migrating from a previous version, meaning that your files will be given a new external ID if you are migrating from version 1. Deprecation of Old Documentum Extractor:By adding support for D2 in the Cognite File Extractor, we are discontinuing our Documentum extractor. We encourage users to transition to the more robust File Extractor. While the Documentum extractor will continue to function as usual, please note that bug fixes and support will no longer be provided. Introducing Public Beta Service for Azure EventHub and Kafka:We're excited to introduce a public Beta service to connect with Kafka. Seamlessly access Kafka messages with minimal configuration, and enjoy full hosted integration service within CDF.Thank you for joining us on this journey of innovation and excellence! Stay tuned for more updates and improvements on the horizon. Warm regards,Elka Sierra  

Related products:Extractors

Data Modeling UI Updates - March Release

Data Modeling provides you with the flexibility to define your own Industrial Knowledge Graphs based on relevant industry standards, your organization‚Äôs own data structures and use cases, or a combination of all of these. Large, and often complex, Industrial Knowledge Graphs might be needed to represent the full extent of your industrial data across disciplines. An important aspect of these knowledge graphs is being able to explore and iterate on both its data and its structure.With this release, we are enhancing the Data Modeling user interface in Cognite Data Fusion to better display the underlying concepts that power your industrial knowledge graph. With this increased visibility into your models, you can have higher understanding and confidence in iterating the structure of your data model.Space centric exploration of containers and views Explore containers and view information for the data model. Data management summary + sidebarSpace centric exploration of containers and viewsExplore the structures that powers your data model by looking at the space they reside in.When in a space, you can choose the container and view to explore their structure, and the data within.¬†Explore containers and view information for the data model.Within the visualizer and the data model editor, you can now explore the definition of the view, and explore the containers that power them.Beside just the raw configuration, you can also explore them more in a more relational manner via the data model visualizer.¬†Data management summary + sidebarIn the data management tab, you can explore data easier with the new sidebar functionality that let‚Äôs you look through each instance and their relationship (via double clicking on the property you want to delve deeper into)Additionally, a new profile view enables you to better understand the totality of your data. Get an aggregated summary of the data, grouping by whichever property you would like.¬†As part of the release, we have also added or fixed the following:Data model statistics - fixed progress bar in summary not showing correct amount of nodes and edges Data modeling - unit directive support Data model visualization - inverse relationships not rendered correctly Data management - instances not visible for views that inherited properties from another view Data management - filters with dates, timestamps and sorting are working as expected Data connection - new excel connection string option via ‚ÄúConnection‚ÄĚ panelLet us know if you have any questions or feature requests regarding the release below!

Related products:Product ReleasesData Modeling

Product Release Spotlight - March 2024 Release ‚Ēā Cognite Data Fusion

Hi everyone! ūüĎčThe next Cognite Data Fusion product release is soon approaching. Together with the return of the sun (for those of us in the icy north), we are excited to announce several new upcoming features across our most popular user interfaces and DataOps capabilities.This post will walk you through selected highlights from the release:Increase efficiency of troubleshooting and root cause analysis with updates to Industrial Canvas Improve productivity in Jupyter Notebooks with enhanced AI co-pilot and collaboration Accelerate solution development with new Unit Conversion and Time Series enhancements Increase robustness¬†of data pipelines with Data Workflows Improve performance of your Industrial Knowledge Graphs with upgraded¬†Data Modeling UIThere‚Äôs also much more to explore and discover. Dive into the latest release notes and uncover all the additional features and improvements designed to take your industrial data journey to new heights.¬†These brand new additions will be available to you on March 5th 2024. We would love to hear your feedback here on Cognite Hub over the coming weeks. We also want to thank all our community members for your contributions so far. We're eager to continue this collaborative process with you, seeking your valuable input on both existing and upcoming product features. We encourage you to continue to submit your Product Ideas here in the community, to help us understand how we can continue to evolve Cognite Data Fusion to fit your needs. Let's keep this momentum going!¬†Here‚Äôs a short video summarizing the release highlightsIncrease efficiency of troubleshooting and root cause analysis with updates to Industrial CanvasWhen troubleshooting complex issues, a lot of time is spent on finding the right data and sharing it across disciplines. Collaborating in a visual, seamless way reduces the time and effort to resolve issues and get your systems back to optimal performance. This release we are introducing new features to Industrial Canvas which improve support of troubleshooting and root cause analysis for our industrial users.¬†In Industrial Canvas, we are enhancing sticky notes so that you can build cause map diagrams for your root cause analysis. Another highly requested feature has been a tighter integration and user experience between Industrial Canvas and Charts. You will now be able to open and view your Charts directly in Industrial Canvas, collecting all essential data in one place, and removing the need to go back and forth between browser tabs.¬†¬†Improve productivity in Jupyter Notebooks with enhanced AI co-pilot and collaborationJupyter Notebooks in Cognite Data Fusion allows you to write and run Python scripts without leaving your browser. The notebooks use your current user credentials when logged in to Cognite Data Fusion, which removes the need for additional authentication steps and risk of getting access to restricted data. Since its initial launch late last year, Jupyter Notebooks has become an increasingly popular feature for our more technical users¬†such as data scientists and engineers, being used to¬†explore data and prepare for building data pipelines and solutions.In this release we are further improving Jupyter Notebooks‚Äô AI copilot. The copilot has received a major visual overhaul making it simpler to use, together with elevated knowledge of Cognite Data Fusion and its Python SDK increasing its accuracy. You will also find an upgraded user interface showing where notebooks are stored. This allows you to differentiate between notebooks stored inside or outside of Cognite Data Fusion, enabling sharing of individual notebooks and prevention of potential data loss.¬†¬†Increase robustness¬†of data pipelines with Data WorkflowsAll data in Cognite Data Fusion rely on industrial data being efficiently onboarded and contextualized, while maintaining the quality and trust in the data. Data managers, data engineers, and data scientists usually manage several data processes to prepare Industrial Knowledge Graphs and other solutions for their organization‚Äôs consumers.¬†The introduction of Data Workflows allows you to orchestrate execution of Cognite Data Fusion‚Äôs data processing features, such as Transformations and Functions. This enables¬† significantly increased observability and control of the end-to-end data pipelines, leading to improved performance and faster scaling.¬†Accelerate solution development with new Unit Conversion and Time Series enhancementsCognite Data Fusion‚Äôs core data resource types, such as Data Modeling and Time Series, are extensively used in creation of Industrial Knowledge Graphs and solutions tailored for solving industry specific problems. Enabled by Cognite Data Fusion‚Äôs DataOps capabilities and SDKs, solution builders are often setting up calculations and similar processes to prepare their industrial data to fit their use cases. In this release, we are introducing a set of new features for Unit Management and Time Series aiming to expand this toolkit. These new features cover several commonly requested gaps in Cognite Data Fusion, which we believe will simplify and accelerate the experience with exploring and consuming relevant data, and deploying solutions on top of it.Building upon our latest Unit Management features, which allows you to assign units of measure to Time Series instances and Data Modeling properties, you will now be able to consume data in the unit of measure relevant to you. This allows you to spend less time on error prone unit conversions, both for simple and one-off data exploration to more complex solution building.For Time Series, we are introducing Subscriptions and Data Quality Status Codes. Up until now, our users have had to download the full range of data points to get updates on new or changed data points. This increases the risk of performance challenges, especially for solutions or processes dependent on large data volumes. The new Subscriptions feature allows you to set up one or more Subscriptions across Time Series, and obtain only the latest and changed data instances that are to be used in e.g. dashboards or calculations. Data Quality Status Codes allows you to represent the data quality of Time Series data points according to the OPC-UA standard. This provides you with the option to choose how to handle data instances not considered good enough, instead of them automatically being removed when being ingested into Cognite Data Fusion from source systems. The added insight into the quality of data instances will increase trust in the industrial data, both for solution builders and data consumers.¬†Improve performance of your Industrial Knowledge Graphs with upgraded Data Modeling UIData Modeling provides you with the flexibility to define your own Industrial Knowledge Graphs based on relevant industry standards, your organization‚Äôs own data structures and use cases, or a combination of all of these. Large, and often complex, Industrial Knowledge Graphs might be needed to represent the full extent of your industrial data across disciplines. This increases the risk of the Industrial Knowledge Graph having reduced performance for its consumers, such as query timeouts and slow response times.With this release, we are enhancing the Data Modeling user interface in Cognite Data Fusion to show how relationships between data is stored, and what can be queried. This allows you to better understand how you can optimize your graph to delivert the best possible performance and stability for your consumers.You can find more detailed information on the updates in the¬†release notes¬†on our Documentation portal.

Related products:Product Releases

Embark on your data science journey with Jupyter Notebooks in Fusion

Jupyter is a tool loved by data scientists, analysts, and engineers, particularly for its interactive computing environment and robust data processing capabilities. This platform allows users to dynamically write and execute code, visualize results in real-time, and efficiently handle large datasets, making it an essential tool for in-depth data analysis and exploration. Using Jupyter Notebooks¬†in Fusion, you can now write and run Jupyter Notebooks in Fusion without leaving your browser. Notebooks can be stored in Cognite Data Fusion (CDF) allowing easy access and sharing. All code is executed in a ‚Äúsandbox‚ÄĚ within your browser, reducing the chance of leaking information. In this sandbox, notebooks do not have access to your local files or information stored on your computer. All communication with CDF and notebook code is executed under the current user credentials, so Jupyter cannot be used to gain access to restricted data stored in CDF.¬†Paired with our new GenAI code copilot, we have made it easier than ever to get productive with CDF. The copilot will help you write and explain code. Soon, the copilot will also help you fix code errors.If you are new to Jupyter, we recommend first reading the official JupyterLab introduction guide before reading the rest of this guide.Join the Beta: Your Feedback MattersJupyter Notebooks in Fusion is in public beta. As with any beta program, some level of instability and unforeseen issues may be expected. We strongly encourage users to actively participate in this beta phase by providing feedback, which is invaluable to us. For product interaction and reporting bugs, please visit Cognite Hub. If you have ideas for improvements you would like to share, please create a new product idea on Cognite Hub. Your contributions are not only welcome but essential in making JupyterLab in Fusion a more robust and user-friendly tool.Using Jupyter Notebooks in FusionThis chapter guides you through the process of using Jupyter Notebooks within the Cognite Fusion environment, detailing how to access, store, and manage your notebooks. You will also learn how to authenticate with CDF to access information stored there and how the new copilot can help you accelerate the process of writing code.Jupyter Notebooks is available under the Explore menu in Cognite Fusion. Once started, previous Jupyter Notebooks users will recognize the familiar user interface. Notebooks and files are accessible in the file browser.¬†StorageThe first time you open Jupyter Notebooks, you will notice a few folders already available:examples/ - This folder contains some interesting examples of things you can do with JupyterLab in Fusion as an inspiration. quickstart/ - A quickstart guide for getting started with data modeling and population of data. CDF ‚ėĀ/ - A special folder for loading and storing notebooks to Cognite Data Fusion.¬†There are two different ways of storing notebooks and data in Jupyter Notebooks in Fusion; locally or in Cognite Data Fusion (CDF). By default, all files and notebooks are stored within the browser storage. Since this storage can be cleared (e.g. as a result of clearing the browser cache) it is recommended to only use this storage for small notebooks that temporary files you can afford to lose. Locally stored notebooks are accessible across projects - that is, you will have access to notebooks from CDF project A when you are working in project B.To store persistent notebooks and share work with your colleagues, use the CDF-folder. Inside this folder, you will find each data set you have read-write access to. If you already have a data set you want to store notebooks in - great! If not, see section below for creating data sets for notebooks. To save a notebook to Cognite Data Fusion, simply place it inside the respective folder in the file explorer in Jupyter. Note that only notebook files with IPYNB-extension will be saved to CDF - no other files will be synchronized.Authentication helpersWhen using Jupyter Notebooks in Fusion, it‚Äôs recommended to use the default authentication helpers provided. To use this helper, simply add the following code to your notebook:from cognite.client import CogniteClientclient = CogniteClient()This authenticates using the current user credentials, meaning that any code that is run in Jupyter Notebooks in Fusion when using the authentication helper has the same access permissions as the user.GenAI code assistantJupyter Notebooks in Fusion comes with a useful coding assistant. This assistant will help you write code based on natural language input and explain existing code. Later it will also be able to help you fix code errors.The code copilot is accessible from the cell editor in Jupyter. You can use the copilot to generate and explain code. Later we will also add functionality to help you fix code with errors.Best practices for succeeding with the code copilot includes:Be explicit. If you want the copilot to work on data you retrieved in previous cells, state the name of those variables. You also might want to hint as to what type of those variables are, especially if it's not obvious from the code in prior cells. Use Cognite Data Fusion lingo to help the copilot understand what APIs are relevant. An example of this is that if you are retrieving work order data, provide information on how to retrieve this data - not just instruct the copilot to retrieve work orders. Instruct the copilot to do one single task per cell. For performing additional tasks, add additional code cells. Manually combine code from multiple cells after verifying that the code works. Write instructions in English. Other languages might work, but expect best accuracy when using English. Always read through the generated code before running to make sure the code doesn‚Äôt perform harmful actions. This is especially important if you have elevated access rights to CDF to ensure data integrity. Expect to be required to change parts of the generated code. The copilot will make mistakes, but can help you point in the right direction.Some examples of prompts you might want to try out is:Initialize a Cognite client Retrieve all root assets Search for documents by "Operating procedure" and print content of the 5 first documents Fetch time series value for the last year for time series with external ID "XYZ". Plot the time series and print min, max and mean values.Note that this is an early version of the code assistant. Expect accuracy to improve as we develop our coding assistant over the coming months.GenAI data insightsWe also introduce a new Python package, cognite-ai, that will help you use GenAI not only as a coding assistant, but also to simplify data insights in your code. This package is based on PandasAI and allows you to ‚Äútalk to your data frames‚ÄĚ from CDF An example notebook is available under the examples/-folder.Note that this Python package also can be used outside Jupyter Notebooks in Fusion. This package is in an experimental state.Security considerationsWhen you are using the default authentication helper, all code executed in Jupyter Notebooks in Fusion is run under the current user credentials meaning Jupyter Notebooks will not provide access to any data the user doesn‚Äôt already have access to.Notebooks are stored in CDF and/or browser storage. Local cached versions of notebooks are kept in browser storage. It‚Äôs advisable to clear notebook cell outputs if sensitive information is listed here.Never store secrets in notebooks, even for private notebooks.¬†Limitations and known issuesKnown issues:Note that due to restrictions in the underlying technology, you will only be able to have one browser tab/window with Jupyter Notebooks open at the same time. The link generated from ‚ÄúCopy sharable link‚ÄĚ file context menu does not work Packages with native libraries are not supported. When you see an error message like this: Can't find a pure Python 3 wheel for ‚Äė<package>‚Äô. See: https://pyodide.org/en/stable/usage/faq.html#micropip-can-t-find-a-pure-python-wheel. You can use `micropip.install(..., keep_going=True)`to get a list of all packages with missing wheels. it typically means that the package requires a native library and cannot be used with Pyodide. See Pyodide document for more information. It‚Äôs not possible to interrupt execution of notebook cells.Advanced: preparing datasets for sharing notebooksIn order to store notebooks in Cognite Data Fusion (CDF), you must have access to reading and writing files to a Data set. To share notebooks within your organization all collaborators must have file:read and file:write ACLs. Access to managing Data sets is usually restricted, and you might need a CDF administrator to help you prepare Data set(s) and manage access.¬†

Related products:Jupyter Notebooks

Product Release Spotlight - January 2024 Release ‚Ēā Cognite Data Fusion

New year - New release! Welcome to 2024, the year that brings easy accessibility and seamless utilization of complex industrial data, thanks to the power of Generative AI.   Easier to get started with the new landing page on Cognite Data Fusion Finding data for industrial users is now vastly improved in the new search experience (including both (F)DM and Asset Centric) Engineers doing collaborative troubleshooting and root cause analysis are supercharged by key features requested by our core users in Industrial Canvas In direct response to your feedback, we’ve improved monitoring of equipment and processes with editing, highlighting and more context in alert emails And finally for all industrial users, first class Units support! And that’s not all, there’s much more to explore and discover. Dive into the latest release and uncover all the additional features and improvements designed to take your industrial data journey to new heights. These brand new additions will be available to you on January 9th 2024. We would love to hear your feedback here on Cognite Hub over the coming weeks. We also want to thank all our community members for your contributions so far. We're eager to continue this collaborative process with you, seeking your valuable input on both existing and upcoming product features. We encourage you to continue to submit your Product Ideas here on the community, to help us understand we can continue to tailor Cognite Data Fusion to your needs. Let's keep this momentum going! Here’s a short video summarizing the exciting updates  Easier to get started with the new landing page on Cognite Data Fusion Working in critical roles we know how important it is to get started fast, and get straight to work. This is why we are launching a new, focused landing page for Cogniter Data Fusion with quick access to Search, Industrial Canvas and Charts. Your most recent work is highlighted for you to get right back where you left off. All of your powerful features of Cognite Data Fusion are available in the menu, with quick access to what you need.  Finding data for industrial users is now vastly improved in the new Search experience With the launch of the brand new Industrial data search experience we are simplifying how you can search, browse and find your industrial data across different types of data. Search and browse across 3D, sensor data, equipment, documents, work orders and more, all in one place. The refreshed experience is optimized for industrial end users looking for quick answers across complex data. This includes built in Generative AI search where you can use natural language to get simple answers to complex questions powered by your industrial knowledge graph.   Supercharge your visual troubleshooting with Industrial Canvas When troubleshooting complex issues, most time is spent on finding the right data and sharing it across disciplines. Collaborating in a visual, seamless way reduces the time and effort to resolve issues and get your systems back to optimal performance. With this release we are introducing more control when collaborating visually in Industrial Canvas. You can lock down root cause analyses to prevent changes after an issue is resolved. You can also roll back to earlier versions of an analysis through automatic snapshots of your work. Enrich your P&IDs and technical documents with a built in symbol library, and export all markups as PDF.Here you will also be guided by Generative AI, where all documents can be summarized and interrogated for questions. Long, technical documents can give you structures answers to your questions, with references to where in the document the answer was extracted. Improved Monitoring of equipment and processes When monitoring critical equipment and processes, you now get more relevant context and information in alerts emails. You can jump straight into the relevant chart and start troubleshooting, or unsubscribe directly from the email. You also have the ability to edit and tune your parameters of ongoing monitoring jobs, improving the overall workflow of monitoring your plant.  First class Units support Across our APIs and Python and Javascript SDKs we now have full support for industrial units through our Cognite Unit Library. All features and examples are described in our documentation and examples.You can find more detailed information on the updates in the release notes on our Documentation portal. 

Related products:Product Releases

Document Parsing - Using generative AI to assist in Data Management

Dear Community,We would like to provide an update on our ongoing efforts to address the challenges associated with unstructured data. We have developed this closely with AkerBP while concurrently identifying areas where such a tool could deliver significant business value. Throughout the process we have identified that this tool could be used in the following applications such as Material Master data management, Brownfield modification, and management of Asset Lifecycle Information. Significance of the Initiative:The primary benefit is a reduction in the time spent on wrestling with tedious data extraction, allowing users to allocate more time to meaningful tasks. Our document data extraction tool not only saves time but also serves as a safeguard against errors often associated with manual input. Furthermore, we are looking to develop tooling and processes to mitigate and eliminate errors generated by the LLM.Issues Addressed:Problem: Manual data work resembling an endless endeavor. Key Value Driver: Time savings  Problem: Errors and discrepancies resulting from a high volume of manual input Key Value Driver: Error minimization Problem: Applications hindered by sparse data Key Value Driver: Establishment of a robust data foundation for applicationsOur focus is on streamlining processes to establish a singular, definitive digital version, eliminating the need for navigating through multiple iterations. This approach adds tangible business value, particularly in terms of time savings and error reduction, benefiting users who rely on accurate data for various applications. The implementation aims to provide enhanced efficiency and reliability in data handling.For more information on the Data Management journey in Aker BP, feel free to watch the webinar here.

Related products:Contextualization

Deprecation Notice for AIR and Introduction of New Monitoring Capabilities in Cognite Charts

Hi CDF Users,This is a reminder notice for the deprecation of AIR. The alpha-level maturity monitoring solution AIR will be deprecated on 2023/12/31 and we will no longer provide support for the AIR after this date.To ensure a seamless transition and to provide you with improved monitoring capabilities, we are excited to introduce our new solution in Charts which is in open beta and is now available to all customers.Key Features of Monitoring in Charts:Enhanced User Experience: Monitoring in Charts has a better user interface, providing a user-friendly experience for proactive troubleshooting and root cause analysis. Integration with No Code Calculations: Our new solution allows for effortless integration with no code calculations, streamlining the monitoring process and empowering users to derive insights without the need for extensive coding.Stability and Reliability: Monitoring in Charts is built with stability and reliability in mind.Scalability: Monitoring in Charts is built on a backend API that is better suited for scaling making it an overall better solution instead of AIR as your monitoring needs grow. We believe that the transition to Charts will bring significant benefits to your monitoring needs. We appreciate your understanding and look forward to continuing our partnership with you as we embark on this exciting new chapter in monitoring capabilities within CDF.  

Related products:Charts

Early Adopter Product Release Spotlight - October 2023 Release

Dear Community,We're thrilled to share the latest enhancements from our October 2023 product release, exclusively available through our Early Adopter Program! Introducing Data Workflows:Revolutionize your pipeline with our new managed process orchestration service, Data Workflows, integrated within Cognite Data Fusion. With Workflows, you gain the power to seamlessly coordinate and execute interdependent processes, including transformations, functions, requests to CDF, and dynamic tasks, all in perfect order and on time. This service is currently accessible exclusively to our Early Adopters. We're eager to hear your valuable feedback! Join the group and request participation here. Explore Industrial Data with Ease:Empowering industrial users, regardless of technical expertise, our Industrial Data Search feature simplifies the process of discovering relevant data. You can uncover detailed results for your industrial context, and refine your search down to specific site/asset details. Further, you can explore data through multiple dimensions like 3D, AI search, and standard list view, equipped with an array of powerful filter options. Dive in and be part of our Early Adopter Program! Join the group and request participation here.We can't wait to hear about your experiences with these new features! You can find details on the full October 2023 product release here. 

Related products:Product Releases

Product Release Spotlight - October 2023 Release ‚Ēā Cognite Data Fusion

We are excited to announce the October 2023 release of Cognite Data Fusion! This post covers some of the significant highlights of the release: Enhanced data exploration AI-powered data exploration Manual contextualization of 3D models Saving no-code calculations Monitoring asset health indicators Jupyter Notebooks with Cognite AI Copilot These enhancements will be made available on October 31, 2023. We would love to hear your feedback here on Cognite Hub over the coming weeks. You can also find more detailed release notes on our Documentation portal, as well as the October Roadmap update on Product Updates. This month, we focus on data onboarding. We also want to thank all our community members for your contributions so far. We're eager to continue this collaborative process with you, seeking your valuable input on both existing and upcoming product features. Let's keep this momentum going! Here’s a short video summarizing the exciting updates Enhanced data explorationWe are happy to share our newest enhancement to the search experience within Cognite Data Fusion, furthering our ambitions to deliver easy access to complex industrial data. This feature provides user-friendly search and enhanced context for industrial users, such as engineers, technical experts, and operational teams. You can narrow your search to a specific site - like an individual offshore asset or a processing facility - to make the search results more relevant. The search results are grouped and presented in an industrial context and can be viewed through a list view and a 3D view with a wide set of filter capabilities across all given data types. You can easily change between layered models to see CAD models, 360 images, and point cloud models.   AI-powered data explorationAs an advancement to traditional search and filtering of data models, users can now search using AI. You can toggle on AI and find data by using the example questions or write your own using natural language. You can at any time adjust the generated filters and also view the aggregates as charts and text summaries.   Manual contextualization of 3D models3D models such as CAD or laser scans don’t always contain the required information to link the model to assets, and current workflows require a fully contextualized 3D model to enable you to really benefit from the power of 3D. If you're adding 3D models to Cognite Data Fusion, you can now do manual contextualization of both CAD models and point cloud models in Cognite Data Fusion. The asset links can be reflected in all Cognite applications using the Reveal 3D viewer, enabling users to maximize value capture from applications such as InField and Maintain or any bespoke customer applications built with Cognite Reveal.  Saving no-code calculationsIf you're using CDF to create and run calculations, you can now create, save, and schedule the calculations in Charts. This eliminates the need to create Python scripts and deploy functions. The scheduled calculations can be consumed on external dashboards or used as input to advanced models.  Monitoring asset health indicatorsMonitoring asset health indicators and performance KPIs is one of the key jobs of engineers and subject matter experts when evaluating asset health. We’re now adding the ability to set up threshold-based monitoring in Charts, using raw time series or the saved no-code calculations. You can also enable email notifications to be sent out when thresholds are reached for the monitoring jobs.   Jupyter Notebook with AI copilotData scientists and data engineers can now work with and run Python scripts from Cognite Data Fusion without setting up infrastructure. This is a valuable tool when combining the power of Cognite Data Fusion with external tooling, such as data analytics packages, and when you need to enrich your data using custom logic. Notebooks run in a sandboxed environment inside the browser under the current user credentials, ensuring data integrity.We’re also introducing a code copilot integrated with Jupyter Notebook. By leveraging GenAI, the copilot will help you write and understand code. Code generated from natural language helps you get to value quicker by reducing the need to write boilerplate code or do advanced data analytics. We now support natural language data insights using large language models in notebooks. This allows you to ask questions about data retrieved from Cognite Data Fusion or to generate plots.  You can find the product documentation here Early adopter programsData workflows is a new managed process orchestration service within Cognite Data Fusion. Using CDF Workflows, you can orchestrate the order and timely execution of interdependent processes such as transformations, functions, requests to CDF, and dynamic tasks. The service is only available by joining the Early Adopter Program. Request participation through the group on Cognite Hub. Industrial data search enables industrial users without technical background to easily find relevant data. Search results show the industrial context with the ability to narrow down the search to a specific site/asset, and with several lenses/dimensions of searching and viewing data, such as via 3D, AI search, and standard search with list view and a range of filter capabilities. Please come and join us in the Early Adopter Program. Request participation through the group on Cognite Hub.

Related products:Product Releases

Cognite Data Fusion October Roadmap Update: Data Onboarding

As fall moves closer to winter, it is again about time to provide a short update on the roadmap of Cognite Data Fusion, and share some of the exciting work that is going on in different parts of the product. To avoid overloading you with information, we will split this update into several sections, allowing us to provide more content but in bite-sized chunks that are easier to digest - and to produce. We‚Äôll be submitting a set of updates in the weeks to come, so please stay tuned for more news on other parts of the product. As always, we‚Äôd love to hear your feedback, questions and ideas right here on Cognite Hub.Data Onboarding deals with the job of extracting data from source systems, loading it into Cognite Data Fusion, and processing the data to build the industrial knowledge graph. This also includes contextualization, the technologies that enable Cognite Data Fusion to find connections and relationships between data even if the connecting data elements are not obvious, and thus build a knowledge graph which is richer than the data it is created from.¬†A rich¬†connector marketplace, and ease of setting up connectorsThe catalogue of systems that Cognite Data Fusion supports with packaged extractors is ever growing - our goal is to hit 100+ by the end of the year. The list of systems that are coming up is too long to describe here, but you can look forward to both support for new systems such as Aspentech‚Äôs Enterprise Historian¬†IP.21, support for a wider range of SAP versions and protocols, as well as significant improvements to existing capabilities for systems like relational databases and Documentum. One fun and suprisingly useful piece of connectivity coming up shortly is the ability to connect directly to Excel and Google Sheets spreadsheets and use these as tabular sources of data. In addition to building a richer catalogue of connectors, we are also working on making it easier to host and run extraction jobs directly in Cognite Data Fusion, whenever the upstream system is reachable from the cloud. Hosted extractors provide a very simple ‚Äúclick and configure‚ÄĚ experience, making it very easy to set up new extraction pipelines directly from the UI of Cognite Data Fusion, as well as providing scalable performance and ease of monitoring and operating the extractors over time.¬†Data Workflow Orchestration - streamline and turbocharge your data pipelinesTowards¬†the end of the year, we will be shipping the first release of a new way of transforming and working with data being onboarded to Cognite Data Fusion. We are moving beyond the current capabilities of the transformation pipeline, and introducing the concept of a¬†data workflow, which contains a set of data onboarding tasks (These can be Spark Transformations, Cognite Functions, or many other tasks)¬†that are performed on the data to be processed, either in parallel or in sequence. These tasks can be triggered by schedules, external events or by previous tasks, enabling data workflows to move from schedule-based workflows to being triggered by updates and proceed to execute a set of tasks efficiently and without delay. This will dramatically increase the capabilities, predictability and performance of the data pipelines in Cognite Data Fusion. Not satisfied with improvements under the hood, we also aim to ship a rich user experience for working with data workflows as part of the product. Look forward to the first iterations of this towards the end of the year, and continuing on into 2024.Conceptual designs for data workflow editor¬†Contextualizing engineering diagramsWe are targeting major improvements to the capabilities of Cognite Data Fusion of¬†extracting context and meaning from engineering diagrams, more specifically Piping and Instrumentation Diagrams (commonly referred to as P&IDs). These diagrams are core for understanding system context of an industrial asset. We already support extracting tags from P&IDs, allowing them to be linked to assets (and thus to timeseries data, 3D models and more), but we are now looking to take the next step, and extract more knowledge from the symbols in the diagram. Our goal is to build a tool that allows you to identify symbols, connections and flows of fluids (or gas) in a system. This tool will be possible to train on the particular standards and symbolic conventions used in your company, and will allow for human-assisted contextualization of P&IDs where you can train the system to gradually do more and more of the work. This¬†will unlock new levels of richness in the industrial knowledge graph, and enable¬†new use cases with the unique data that will only be available in Cognite Data Fusion. Look forward to the first updates and a richer contextualization tool for P&IDs early in 2024, with subsequent improvements coming later in the year.¬†¬†Example engineering diagram showing rich data contextualization¬†Onboarding data from documents using Generative AILastly, no roadmap update is complete without mentioning at least one intiative fueled by the rapid evolution of generative AI. This time around, we‚Äôd like to highlight some work we are doing on leveraging generative AI to extract data from documents, and onboard that data into Cognite Data Fusion. This is an application that has many use cases, a simple one being¬†the ability to create a digital representation of PDFs containing¬†equipment data sheets, where the values specified (for things like maximum and minimum operating temperatures, pressures, nominal power consumption, etc)¬†can¬†then be used¬†as actual data (complete with lineage, units of measure, etc.) inside Cognite Data Fusion in things like monitoring jobs, analysis, troubleshooting and data science. In other words, onboarding actual data from a PDF file will feel just as simple as onboarding data from a table in a database. Look forward to seeing the first product releases with support for leveraging generative AI to onboard data from documents in early 2024.¬†If you want to follow our development and shaping of these and many other features of Cognite Data Fusion, use Cognite Hub to engage with us and stay up to date with the latest developments.¬†Cognite Roadmaps are forward-looking and subject to change.

Related products:Product Roadmap

Product Release Spotlight - August 2023 Release ‚Ēā Cognite Data Fusion

We are very excited to announce the August 2023 release of Cognite Data Fusion. This post covers the highlights of the release, which will be available to our customers on September 5th, and includes:  Infield - productivity and GenAI upgrades Maintain 2.0 - new version with improved collaboration, productivity, and visualization features Industrial Canvas - collaborative troubleshooting Extended language support across our user interfaces Power BI connectivity - data modeling support In addition, you can expect an updated version of the Cognite Data Fusion roadmap to be shared in September. The roadmap will provide you with an overview of our plans for the remainder of 2023 and well into the next year.We also want to thank all our community members for your instrumental role in shaping this release. Your contributions and collaborative efforts right here in the Product Ideas section on Cognite Hub have been vital in driving the development of our product. Thank you for being the driving force behind our continuous improvement journey! A short video summarizing the release  Infield - productivity and GenAI upgradesToday, field workers mostly use paper-based documents to perform their tasks and gather data, which is both time-consuming and often leads to the data not being captured in any digital format for re-use. Infield now offers creation of digitized checklists with increased flexibility and automation, with all data captured through the application made available in Cognite Data Fusion. This allows for re-use and improvement of data quality. We are also introducing a new Generative AI feature in Infield. You can now use AI to assist you in searching for information in documents, for instance an operating limit for a pump, and get the result in seconds instead of manually scrolling through tens of pages in manuals.AI powered document search & checklist creation in Infield Maintain 2.0 - improved collaboration, productivity, and visualizationPlanners meet several challenges in their everyday work, including collecting and keeping track of which data to use, finding opportunities to execute additional work orders, and optimizing the productivity of available resources. Maintain 2.0 offers users a significantly improved collaboration experience, including sharing and commenting functionality. To improve productivity, work orders can now be mapped to different grid areas in the two-dimensional layout diagrams, in addition to enhanced automation of creating plans. We are also introducing improvements on how to visualize activities, with the choice of highlighting them directly in the plan or Gantt chart itself, in two-dimensional layout diagrams, or 3D models representing the facility. A new overview of resource allocation over time will also help planners and coordinators in optimizing their plans. Sequencing of activities across different lenses in Maintain Industrial Canvas - collaborative troubleshootingThe Industrial Canvas is a no-code tool that revolutionizes data exploration and visualization. It is a collaborative and visual workspace to work across documents, engineering diagrams, sensor data, images, 3D models and more. It gives you simple access to complex industrial data, and allows you to troubleshoot, collaborate and act on insights across multiple disciplines. The latest addition to Industrial Canvas is that users now can leave comments and tag colleagues in the canvas they are working in, which triggers an email notification to the person being tagged. In addition, canvas visibility can now be set to either private or public. This allows you to work in a safer environment with colleagues when troubleshooting and performing root cause analysis, before ultimately opening up and sharing your findings with the broader organization.Commenting, user tagging, and sharing in Industrial Canvas Extended language supportOur most popular user interfaces, Charts, Industrial Canvas, Data Explorer, Maintain, and Infield, are now being made available in up to 10 languages. This will help our non-English language users with more effectively utilizing our product, contributing to faster onboarding and increased productivity. All users can select their preferred language by clicking on their profile in the upper right corner.Language selection in Infield Power BI connectivity - Data Modeling supportOur Power BI connector has been upgraded to now also include data from our recent Data Modeling feature. This enables users to load and filter data from their own customized data models, allowing for much richer and specific representations of their industry facilities, processes, and similar. This data can as before be used in the creation of Power BI dashboards and reports. Power BI dashboard using data from Data ModelingWe know many of you will find these updates useful and valuable. If you have any feedback or ideas, we’d love to hear from you right here on Cognite Hub!  You can find the documentation here 

Related products:Product Releases

Product Release Spotlight - June 2023 Release ‚Ēā Cognite Data Fusion

We are very excited to announce a brand new release of Cognite Data Fusion. This post covers the highlights of this release coming to our customers on June 20th, and guides you to where you can go to get more details. Please drop a question, start a conversation with peers or give us product ideas here in our community on Cognite Hub.A short video summarizing the release New landing page It can be overwhelming to get started when logging in to Cognite Data Fusion, given that a modern Industrial Data Cloud platform has so much to offer. With all the different possibilities of Cognite Data Fusion, you would prefer to start with a clean and simple landing page guiding you to what is relevant for you. The next time you visit, you will find recently opened tools and recent activities to jump right back in where you left off. This allows for easier onboarding, and useful resources such as documentation, online training courses, support and our community is easier to access.  Industrial CanvasFor early adopters we are introducing a powerful new capability in Cognite Data Fusion. The Industrial Canvas is a Generative AI powered, user-centric no-code tool that revolutionizes data exploration and visualization. A collaborative and visual workspace to work across documents, engineering diagrams, sensor data, images, 3D models and more. It gives you simple access to complex industrial data, and allows you to troubleshoot, collaborate and act on insights across multiple disciplines. A collaborative way of piecing the data together, into context, powered by the industrial knowledge graph.  Extensions to advanced data modelingTo realize the full value of the new data modeling capabilities that was released in April, you need them to extend to the wider ecosystem of Cognite Data Fusion. With this release we are making it easier to get started with out-of-the-box data models. The first one available is CFIHOS, which is an industry information standard that aims to standardize across both operators and equipment manufacturers and suppliers. You can also build operational dashboards and share insights based on powerful industrial data models through updated connector to Grafana. In addition we will start making native data exploration for these new data model capabilities limited available to selected customers.SAP extractorsERP data is an important source of data for solving asset heavy industrial problems, improving asset performance by integrating work orders, maintenance data, asset information etc. to your industrial knowledge graph. Creating and customizing connectors to SAP data can be time consuming and manual. This is why we have productized a robust SAP connector that is available for download and documentation through the Cognite Data Fusion user interface. It supports standard SAP OData interfaces as well as custom OData services built on SAP Gateway across all SAP modules.User profilesEnhanced collaboration in digital tools require that users have profiles where they can store basic information to allow advanced sharing, tagging, commenting and further collaboration features. In order to make this possible, Cognite is making user profiles available in Cognite Data Fusion, integrating with your Identity Provider, creating the foundation to enable advanced sharing and collaboration within the tools in Cognite Data Fusion.In-browser Jupyter Lite notebooks (Limited Availability)Getting started with coding on industrial data can be a big hurdle. Restrictive policies, lengthy setup and configuration including hosting, maintaining and updating over time. This is why we are launching for early adopters in-browser Jupyter Lite notebook Pre-authenticated to Cognite Data Fusion through the user interface. It will pre-load the latest SDK each time you log in, fast and convenient to get started with coding in one click. No time wasted setting up environments, loading SDKs or spending time on authentication. This will also ship with a set of  examples of typical use cases to get started.  We hope you find the updates useful and valuable. We’d love to hear your feedback in the comments! You can find the detailed documentation here

Related products:Product Releases