Product User Community

Hi, I wrote a short article some time ago about how data-driven solutions are playing a critical role in helping companies meet their sustainability goals. Here are some key highlights from the article: Sustainability is becoming increasingly important for companies, who are turning to data-driven solutions to monitor, report, and reduce their environmental impact. Cognite has encountered many innovative solutions that promote sustainability, such as automating greenhouse gas emissions reporting and using robotics to detect dangerous leaks. A large set of sustainability metrics in industrial settings are best managed by real-time calculations, which need to be packaged as trusted data products for consumption. Diligent and accurate metrics are important, but decisive actions matter most. With a solid data foundation, companies can optimize operations and drive positive impacts in sustainability. Cognite Data Fusion enables companies to deliver positive impact supporting their sustainab

The cognite replicator fails to replicate asset data and the behaviour is inconsistent. It worked yesterday and not working today. Note that all resource types like time series, events and datapoints are replicated consistently. I am getting below error while replicating the asset data.Please see attached screenshot for the successful run of the asset replication yesterdayTraceback (most recent call last):  File "/Users/j.subhash.parandekar/oid-replicator/oid_replicator/cognite_replicate.py", line 84, in <module>    assets.replicate(SOURCE_CLIENT, DEST_CLIENT, config=cognite_config,  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/cognite/replicator/assets.py", line 324, in replicate    src_dst_ids_assets = create_hierarchy(                         ^^^^^^^^^^^^^^^^^  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/cognite/replicator/assets.py", line 220, in create_hierarchy    updated_assets = replication

In this post we’ll share with you our thoughts on key areas to focus on to empower you as Domain Experts - frontline workers - to efficiently utilize data and analytics in day-to-day operations. Let us know in the comments what you think!  The direction for Cognite Data FusionCognite Data Fusion (CDF) is an Industrial Data platform, and part of our mission is to ensure that we provide a “batteries included” experience to quickly realize value across our target industries Manufacturing, Energy and Power and Renewables.We want CDF to be the go-to tool for not only Data Scientist and deeply technical roles, but also the Domain Experts that know first hand what the operational challenges are and what the optimization potential is. Over the last couple of months, and in our upcoming releases, you will see a focus on enabling you as a Domain Expert to solve industrial data problems through low-code user interfaces - without assistance from “a coder”.  What sort of industrial data problems ar

User sessions are managed via your IDP. Access token lifetime can vary from 60 to 90 minutes.So once the session expires, the user would normally have to sign-in again. How to overcome this?With the OIDC workflow, it is possible to retrieve a new access token without prompting the user to provide credentials again. This is done by finding a valid access token from cache or by finding a valid refresh token from cache and then automatically use it to redeem a new access token. The diagram below shows the normal OIDC workflow: Below you can find a sample code snippet which uses the acquire_token_silent method available through the class: msal.PublicClientApplication:def authenticate_azure(app): accounts = app.get_accounts() if accounts: print("Taking the token silently") creds = app.acquire_token_silent(SCOPES, account=accounts[0]) else: print("Taking token interactively") creds = app.acquire_token_interactive(scopes=SCOPES, port=PORT) return credsY

A common setup in CDF (Cognite Data Fusion) is to first build an asset hierarchy, and then to attach events, timeseries etc. to those assets. One way to do it is to use transformations. In this short article, we’ll take events as an example. It works the same way for timeseries and sequences. Let’s assume we already have an asset hierarchy and we want to attach events to it. We’ll suppose that we have a RAW table, from which we want to create events, that looks like this:   In transformations, to attach an asset to a created event, you need to specify the corresponding asset ID (assetId in the target schema). Asset IDs being automatically generated, we usually prefer using external IDs because we know what they are made of: an ID from an ERP, a value following a naming convention, etc.  Since the asset ID is the expected value for the assetId field in the target schema, we need to retrieve for each asset its ID, based on its external ID. As mentioned in the documentation, we can read f

The coordinates of a document in CDF are normalized. In a normalized document, it has (0,0) coordinates in the upper left corner and (1,1) coordinates in the right bottom corner.Convert xml coordinates to normalized coordinatesTo convert xml coordinates to normalized coordinates you need to divide X coordinates with the width of the page and the Y coordinates with the height of the page. If the xml Y coordinate is 0 at the bottom of the page and the <height> is at the top of the page, the Y coordinate should be flipped as below.Normalized Y = 1- Normalized YOnce the coordinates are flipped;NewMinY = 1 - OldMaxYNewMaxY = 1 - OldMinY Example scenarioIf a user would like to add manual annotations to a P&ID file in CDF the user should convert the xml coordinates of the file to normalized coordinates. 

Hello from the Solutions Portal team!We have a new release coming, which is by far our largest release yet! We hope this release will make it easier toBrowse through your data inside your CDF Project Enable new solutions built on top of CDF Enable new no code solution possibilities  We have a new home screen with new functionality!Any user can now ‘Browse solutions’. Through here, you can view all solutions that Cognite has to offer. If they’re not enabled, a demo can be requested. If you are an admin, SOME applications can be installed through a wizard for free (today, blueprint) and some are automatically installed (Charts).An explorer designed for SMEs. We’ve taken the old ADI explorer, made some improvements based on feedback, and now have an early version out for early adopters.An early version of global search is included also - easily search your way through any CDF resource (with more resources to come!) New CDF application: Blueprint!Blueprint is a no code solution that allows

Digitalization PoCs are commonplace. Real return on investment (ROI) isn’t.So how do investments in digital transformation efforts translate into real value for your business (or company)?We, at Cognite, commissioned a study from Forrester Consulting to examine the potential ROI and business benefits asset-heavy industrial organizations can expect from deploying Cognite Data Fusion. Forrester interviewed six representative customers across our customer base in Oil and Gas, Manufacturing, and Power & Cleantech with experience using Cognite Data Fusion, and found $21.6 million in added net present value at a 400% ROI. Key results of the ROI study include:* $9 million gained through the optimization of heavy machinery and industrial processes* $5.1 million saved through optimized energy use and reduced operational costs* $4.8 million added value through reduced shutdown time* $4.3 million cost reduction by optimizing planned maintenance  Do you want to know what stands behind these nu

I have set up both the environment variables SOURCE_CLIENT_SECRET (Note that the Client secret is generated from the OID widget as per the documentation)and DEST_CLIENT_SECRET and running the replicator with the config file using option 5.I am getting error as per below logAMAC02Z3123LVCJ:oid-replicator j.subhash.parandekar$ poetry run python3 ./oid_replicator/replicate.py 2023-02-17 18:46:48,239 root INFO - Config file - Repeat line 5: 2023-02-17 18:46:48,239 root INFO - Config file - Repeat line 14: 2023-02-17 18:46:48,239 root INFO - Config file - Repeat line 23: Starting replication of resourcesReplicating assets...Traceback (most recent call last):  File "/Users/j.subhash.parandekar/Library/Caches/pypoetry/virtualenvs/oid-replicator-cwaK-6ym-py3.11/lib/python3.11/site-packages/cognite/client/credentials.py", line 364, in _refresh_access_token    token_result = self.__oauth.fetch_token(                   ^^^^^^^^^^^^^^^^^^^^^^^^^  File "/Users/j.subhash.parandekar/Library/Caches/py

I am running the notebook for my own Cognite project as per the example https://github.com/cognitedata/dshub-tutorials/blob/master/advanced/Comparing%20Entity%20Matching%20models%20with%20SDK%20demo.ipynbI am getting below error ModelFailedException: EntityMatchingModel 3531921825900910 failed with error 'JobFailedException: AttributeError('TfidfVectorizer' object has no attribute 'get_feature_names')' at line  results = model.predict(sources=time_series_test, targets=assets).result

Hi, please help me to clarify on the below details as our SLB team is aiming to setup a project with CDF to do some features,Sandbox tenant for SLB  How to connect to production data Embed Python preprocessing script into the Cognite pipeline

Hi, I am trying to transform data from Raw to timeseries in CDF. I made a query and everything seems ok, but when I use distinct to check the external id uniqueness, I get error message:"Cannot have map type columns in DataFrame which calls set operations(intersect, except, etc.), but the type of column metadata is map<string,string>; I don’t know why I get this message, Isn’t it possible to use distinct when wi do MAP or To_Metadata mapping? 

I am the Head of the Services delivery team in Cognite. The team I lead develops and delivers domain use cases based in CDF to our customers in O&G, manufacturing and renewables. I am going to focus here on what are the consequences of corrosion of industrial assets on the economy and the environment, as recently covered in a series of publications (see references section). The economic and environmental cost of corrosion is a significant issue and is much more severe than one could think. While several calculations methods have been used in various studies it is generally accepted that the direct costs of corrosion to the economy are equivalent to roughly 3–4% of a country’s gross domestic product (GDP) [1,2]. The main cost impact is due to the replacement of corroded steel in failing infrastructures (roads, railways, etc.) industrial equipment and buildings. As well, it has been qualitatively estimated that between 25 and 33% of the annual steel production is destroyed once in se

As communicated in this post in December, Cognite Charts is now integrated as a part of fusion.cognite.com. We are planning to retire the stand-alone url charts.cogniteapp.com to have a uniform product experience and single login. Starting February 7th 2023 you will find a message if you access the stand-alone url with instructions on how to login to fusion.cognite.com. You fill find all your charts and data intact, and Charts working as you are used to. There will be a grace period for 2 weeks after February 7th giving room for users to still access their charts from the stand-alone url if you need support to login to fusion.cognite.com.  For those of you who are linking to Charts in your third party applications, Microsoft Teams, etc., this will still function as expected. We will be working on an automatic redirect for coming releases, however still recommend you update the hyperlinks to point to your-organization-name.fusion.cognite.com already now. If you have questions on this, d

When working with CDF or Azure, sometimes I got some errors that were hard to figure out. Experience no doubt will help me figure them out faster, nonetheless this is an important part in enabling usability.

Hey community we are trying to be more active in this cognite hub page and add content consistently from a month to month basis. This will include a series of articles on alerting and monitoring as well as plans regarding where AIR is going into the future.Why Alerting?Automated alerting is an essential part of monitoring. They allow you to spot issues with equipment groups, time series, data quality, pipelines etc.But alerts aren’t always as effective as they could be. In particular, real problems are often lost in a sea of noisy alarms. In short:Alert liberally meaning its ok to spam users rather than not alert at all Make sure that the user has the complete control of how and when they want to be alertedInherent challenges in AlertingSensitivity: Overly sensitive systems cause excessive false positive alerts, while less sensitive systems can miss issues and have false negatives. Determining the correct alerting threshold requires ongoing tuning and refinement. Fatigue: The common ap

 I faced some isuees regarding connection with Azure when working with CDF. I believe there is an area of opportunity there! 

@Savieth Aceros and @Gina Esperanza Jimenez Morales - hope you find the learning paths helpful! Are you taking part in a bootcamp as well?

ContextThis write up describes a basic set up of writing data points to Cognite Data Fusion using Apache NiFi. The source data is power consumption readings from the HAN interface of a power meter in a residential fuse box:The starting point here is an existing MQTT broker that receives data from an existing MQTT client device. We will use Apache NiFi to consume the MQTT messages, extract the readings and write continuously to CDF.The end goal is a live updated time series within CDF with power readings every 2.5 seconds that can be used for analysis or automation within CDF or simply visualization: NiFi flow overviewThe picture above shows the complete NiFi flow. From left to right, consuming MQTT messages, transforming and writing to Cognite Data Fusion. Approximately 120 data points per five minutes, which corresponds to the power meter outputting one reading every 2.5 second. The NiFi flow makes use of the following Processors: Processor  Purpose ConsumeMQTT

As 2022 comes to a close, we thought we’d take a moment to reflect on all the improvements that have been put into InField this year. From small bug fixes to big feature releases, from requests from Hub to user research sessions in person, many of you have contributed to make InField the best it can be for all its users. Throughout the year, InField has been updated with a total of 30 updates packed with user-centered functionality and improvements – and at the same time, the InField team is rebuilding the foundational backend, which sets us up for 2023 and beyond.  Users can expect to see major improvements to performance, much improved 3D- and file-viewing, better scheduling of checklists and more capable templates. While we’re working on that, here are some of the improvement and additions the team has made in 2022: January: The Weekly Summary page was introduced We updated the progress indicator to a new status bar, and made it available on mobile Much improved formatting on de

Below we have outlined several frequently asked questions and their corresponding answers.Don’t see the answer to your question? Post as a reply in this thread and we’ll be sure to answer and/or add it to the FAQ list below!Protip: Use [cmd+f] or [cntrl+f] to search for keywords related to your question.Be sure to also let us know what else we can help answer here in the comments below! Q: What is the plans with Templates going forward?Please check out the documentation hereor https://pr-1023.docs.preview.cogniteapp.com/dev/guides/data-modeling/get-started#templates-future Q: Will there be new features built on top of Templates?Yes and no. We will not be doing any development of backend features (like new capabilities) into Templates. However, we will actually be building a lot of additional tooling on top of Templates (better UI, visualization layers, code generation, CLI etc.). Stay tuned in this Cognite Hub to see how you can use these tooling. 

Cognite Team - thank you for bringing back the ability to overlay events directly on the chart as was previously a capability in ADI. It’s a welcome return and valuable add to the application.After utilizing the app a bit, I have a couple of “low hanging fruit” recommendations:Can we add the event description to the metadata that’s shown on the event? Is there a way to toggle using both start time and end time vs. just using start time? Some of the events that we have run for months and the start time is really the only relevant point for the trendHere are some more complex recommendations:Since charts is now integrated directly in Data Explorer - is there a way to click on the event results and actually go to the event directly in CDF to look at all the metadata? When filtering for events by selecting the asset - is there a way to toggle “select all children” as well? An example is that we have several pumps where work is tagged to the smallest component, but if we filter the events t

Today is the December release of Cognite Data Fusion. As you might have seen in last week’s Product Tour or read in today’s release post, there are several new and exciting Charts-specific updates in the latest release, namely...Charts is now an integral part of Cognite Data Fusion (fusion.cognite.com).A long awaited improvement, charting and no-code analytics finally lives alongside all other industrial DataOps functionalities available in Cognite Data Fusion. No URL switching and additional sign-in required.For those of you who have been using Charts before, all private and public charts that you and your colleagues have created will still be available via Cognite Data Fusion (fusion.cognite.com).Note that charts.cogniteapp.com will still work and be available. This teams, for those of you who are linking to Charts in your third party applications, Microsoft Teams, etc., this will still function exactly as expected. We will be working on an automatic redirect for coming releases, how

Hi, there! At Cognite, our customers are at the center of everything we do. Our goal is to help you to solve the industrial data problems daily. It's all about you & your industrial data.  To make sure we're giving you the best experience possible with the Flexible Data Model (FDM), we'd like to learn about your FDM expectations. Please spend a few minutes to share your thoughts with us in this short survey. The information you give will be confident, so be honest. We really want to know what you think. The survey 👉  https://forms.gle/d8BzpW4jmG4YN9kSA Thanks for your help! Cheers,On behalf of the FDM team   

Hello, I found very interesting that a lot of the conversation around digital has moved away from “AI will change the industry” to much more elaborate analysis on the importance of data in context and data orchestration/DataOps. This realization is making its way all the way to the board room.A good example comes from the recent conversation between Blake Moret (CEO of Rockwell Automation) and Olivier Le Peuch (CEO of SLB). Both companies have independently decided to partner with Cognite and you can find some of the rationale in their exchange below.Two additional notes from the discussion:There is a vision to move from an equipment centric digitalization to a full asset/system view of what digital should encompass - with automation/closing the loop as a key actuator (see Sensia’s control systems’ role),  The discussion around optimization has stopped being solely about production volumes but now also systematically includes carbon. Source: Partial transcript from Rockwell Automation'