Community Discussions

Hey there! I am Sunil, Product Manager for Data Integrations. Since this is our first post in the Hub, let me introduce my team and what we do. My team is responsible for the Ctrl+C Ctrl+V of your data from source systems to Cognite Data Fusion . But on a serious note, we are responsible for ELT - Extract, Load, and Transform of data. We are thrilled that you are interested in learning about the Transformations Python SDK. Yayy!In this post, I list a couple of use-cases/scenarios where the python SDK for transformations makes managing and orchestrating transformation jobs simple and seamless. Our goal is to make the transformations SDK more developer-friendly and would like to hear your feedback. Do check out the API and SDK docs for more details, but now on to a few use cases! Use Case 1: Triggering TransformationsCognite Data Fusion helps liberate industrial data silos and provides a unified and contextualized view of the data. It is now easy to derive insights and build application

If you’ve been feeling like that calculations in your Charts are running faster than before… they are!Yesterday, we released a new version (V2) of our calculations backend to production. This update includes  numerous performance improvements, bug fixes, and quality improvements.In short, it’s an all-around better experience built on higher quality code.Test it for yourself on your CDF project. As always, let us know in this group if you have any questions or feedback!

Thank you again to everyone who joined us live for yesterday’s Product Spotlight webinar! To those of you who couldn’t make it to the live event or if you want to re-watch, you can take a look at the recording, below.If you want to read about the the details of the October CDF release, you can visit the post in Product Updates by clicking here. We already received some great feedback and we’re happy to hear several of you found the product demo with a practical use case example to be helpful. If you haven’t already submitted your feedback about the webinar, please do so here. We’re reading and discussing each response so we can make all of our future product webinars as valuable as possible. We’re already planning a new event about Charts, exclusively available for all of you in our Early Adopter community, so stay tuned for updates and invitations.Have a great rest of your week, everyone!P.S. We also appreciate everyone’s patience with the technical difficulties that were experienced

My name is Sigrid Schaanning, and I have written this post together with Atussa Koushan. We both work as Software Engineers in Cognite’s Disruptive-MVP team, which focuses on robotics and computer vision. In this post, we will look further into how to develop a computer vision model, from gathering data to deploying the model on a robot. First, let’s set the scene for what we are trying to achieve in this post.Imagine performing routine inspections at an industrial facility or an oil platform. You might have to read and note down the values of multiple gauges three times a day - every single day. Or, you might have to inspect different parts of the facility to check for corrosion or wear damage.Routine inspections could be tedious and repetitive work, and it might take place in surroundings which are dangerous to humans.Robots, on the other hand, have no perception of tedious work and they could also operate in hostile environments. Robots, together with computer vision, constitute a p

Hello Charts Group! On November 3rd 15:00-15:45 CET, we will be hosting a live webinar to present highlights from the latest CDF release, with a special focus on CHARTS AND THE CHARTS EARLY ADOPTER GROUP.You will learn about why we’re building Charts, what problems it can solve, and how to use key features and functionalities. We’ll demonstrate how to leverage this new CDF functionality as we build and solve real use cases in Charts step by step. From finding the relevant data, to using P&IDs to gain a systems understanding, to configuring the chart, to creating the no-code calculations to solve the problem at hand. This will be an interactive session for asking questions, discussing ideas, and providing feature requests directly with the team. By the end of this session, you’ll feel confident about using Charts in your day-to-day work and feel empowered to think creatively about the solutions you could build to shape a safer, more efficient, more sustainable industrial future.You

Hello Digitalization Community! On November 3rd 15:00-15:45 CET, we will be hosting a live webinar to present highlights from the latest CDF release, with a special focus on CHARTS AND THE CHARTS EARLY ADOPTER GROUP.You will learn about why we’re building Charts, what problems it can solve, and how to use key features and functionalities. You’ll also have the opportunity to join our private early adopter group for Charts. By the end of this session, you’ll feel confident about using Charts in your day-to-day work and feel empowered to think creatively about the solutions you could build to shape a safer, more efficient, more sustainable industrial future.You can RSVP for the webinar via the link below:This is the first webinar in our Product Release Spotlight series, so you can expect many more after our bi-monthly CDF releases in the future. See you on November 3rd at 15:00 CET!

Hi Community Friends! We thought it might be nice to know who’s who, while we’re actively engaging in our community. I’ll start! My name is Anita, and I’m your Community Director. I joined Cognite in September 2020, and jumped right on the mission of establishing Cognite’s customer community. I’ve worked in a variety of industries, and am passionate about working at the crossroads of sustainable business development and digital transformation. I live in Lommedalen, Norway (translated “the Pocket Valley”), where we have long winters with deep snow. I love horseback riding (Islandic horses in particular), and I have a passion for Italian wine and food.Here’s a fun fact: my grandfather’s brother, who was raised in the remote mountains of Hallingdal in Norway, worked as a chemist with Thomas Alva Edison. The letters he wrote home made me realize from early on that no matter your background, you can change the world. I’m super excited to facilitate for all of you to meet, discuss and learn

Hi everyone,We in C4IR Ocean are starting a series where we are challenging the community to model a certain problem in CDF. The aim with this series is to facilitate discussion and invite community members to share interesting solutions and techniques.The first challenge focuses on Open LineageOpenLineage is an Open standard for metadata and lineage collection designed to instrument jobs as they are running. It defines a generic model of run, job, and dataset entities identified using consistent naming strategies. The core lineage model is extensible by defining specific facets to enrich those entities.In C4IR Ocean, we are dealing with data from multiple providers. Some datasets are open and publicly available, others are closed. It is therefore important to keep track of where data is coming from, and what transformations have been applied to the data after it is read from the provider.We are seeing a good landscape of data lineage solutions - both open and closed source. At the sam

One of the main objectives of the first deployment with an inspection robot is getting your organization ready for robotics. A properly scoped deployment will contribute to engaging the field workers and provide insights on what it takes to keep robots operational on a day-to-day basis. This will lay the foundation for high return on investment for all the following deployments. If the scoping fails, there is a risk of losing trust in robots and gaining less willingness to change in your organization. But fear not. In this post, we’re summarizing our best practices when assisting our customers to scope out their first deployment of an inspection robot.    Selecting the appropriate area of deployment It is common that there is a need to deploy robots in harmful or remote environments. These can be areas that may be highly corrosive, have a risk of explosive gases, have powerful magnetic fields, radiation from heat or nuclear etc. When selecting the appropriate robot for a plant, we reco

The majority of the hydropower fleet is pretty old. It’s not uncommon to see a 100-year-old power plant still in operation. While it may be a great piece of engineering, how can you access operational data on your laptop or smartphone? After all, you may not want to travel every day to very remote locations. At Ringerikskraft’s Hønefoss II power station in Norway, we installed a set of cameras to passively record alarm events, read gauges, and stream this information to Cognite Data Fusion. It is all connected, so if an alarm goes off, I get an SMS notification and can quickly look up what is happening. Pretty cool, or what? I would like to hear your thoughts. How have you tackled the challenge of digitalizing aging infrastructure?Read the full story 

Hi, When developing with the Cognite python SDK, a common restriction is  the API imposed restrictions on queries. Quering time-series, for example, is restricted by 100 asset-ids.  I believe the Python SDK should recognize these constraint violations and batch / concurrently dispatch requests in chunks that do not violate constraints. Optionally with a performance-warning to the developer.Is this sane, or do you think that the developer/customer should maintain a wrapper on the SDK for batching each endpoint (as we’ve currently done at Statnett)?

Hey!In our tooling for the power-analyst we make computations using the SyntheticTimeseries API. Results from these are used in subsequent analysis, where we have identified a problem for us.    When a synthetic timeseries is computed with a specified aggregate and resolution, that specification is returned  irrespective of there being data on the originating timeseries. In our case, we compute aggregates over long stretches of time which results in situations like the image below: In the figure, the opaque line is the comptued by addition of the two other signals. Spanning over roughly a year, the period in the middle has a linear rise in the SyntheticTimeseries while the originals are empty. These values are of course meaningless and should be omitted in the successive steps of the analysis. Have you considered implementing a density filter or the like for these types of situations? Or, do you believe this is best solved client side by identifying the holes prior to a set of Syntheti

 McKinsey Quarterly did post an interesting set of articles recently, regarding the transition and change management when looking into digital services. I believe some of the topic and views could be interesting to reflect upon. Go here to find out : https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/five-fifty-so-you-want-to-be-a-software-company?cid=other-eml-alt-mip-mck&hdpid=034603b3-3fba-4313-a963-553dca543b0f&hctky=12386931&hlkid=8635b3c5c91d439a8a47f0bf353a4968  

How a DataOps platform can boost improvement in all parts of the business! And this time is focusing on what is most important - HSE!https://www.pgs.com/investor-relations/ir-news-stock-announcements/pgs-digitalization-initiative-improves-crew-safety/https://www.cognite.com/customers_stories/dataops-in-action-improving-vessel-crew-safety One more big shout out to @Cerys James and @Sverre Olsen ! Congratulations for another amazing use case!

 It's been a while, but as promised, here is the follow-up post to my previous article. If you haven't read the previous article, you can find it here.In the first part, I shared the basic concepts about data integration. Today I will continue on the same theme, but the focus will be on latency and frequency. They are related concepts, but not necessarily the same. Why latency and frequency? They are crucial to defining your data pipeline, and they influence which Azure resources you use to cover your needs.What is data latency and frequency? Data latency is how fast/slow data can be retrieved or stored. Low latency means that the data is available in real-time or close to real-time and is vital in use cases where you need to respond quickly to information. Examples are alerts for critical events on machinery and equipment and online games based on real-time experiences. This leads us to the data frequency concept. Data frequency is how many times in a specific period the data should b

Hi, my name is Kine Årdal, and I have the privilege of being the Customer success manager for Neptune Energy. I’d like to share experiences from our latest project, and hoping to get feedback and hear about similar experiences from you in the community. The project kicked off in April, as a12-weeks proof of concept, where we enabled a collaborative environment with seamless data flow of key well data for well planning.Current workflow: In the current well planning workflow, it is challenging to share data efficiently and reliably across disciplines and collaboration partners. There is no central storage of drilling and well data ready for a digital way of working, and data is often stored in silos. In addition, the collaboration with vendors often relies on manual exports/imports, sharing data on e-mails/ftp, which leads to e.g. unnecessary time spent waiting for data, duplication of data and a challenge to keep track of versions.Result: In this project, the initial Neptune well data m

Hi all,take a look at https://properate.com/, the landing page for Properate. This is the solution Energima have built on CDF. Remember to watch the landing page embedded video :-)Energima is taking full advantage of several CDF services and Properate includes a stack of cool features!  

The happiest moment in a CSM day/week/month/ YEAR is when we see our customers sharing success cases! Thank you PGS for allowing us to be part of your digital journey! Contratualizations for another fantastic achievement @Cerys James @Sverre Olsen !! 

Database migrations - assets and filesCognite plans to migrate data from Cloud SQL databases in our Google Cloud-based CDF clusters in a two-week period starting June 25, 2021. We plan to migrate the databases containing data for the assets and files resource types. The migration for the two resource types will occur on different days.  We will email customers the exact time of the maintenance window for their cluster, CDF project, and resource type.  Why is this database migration needed?Cognite has recently made CDF available on Azure in addition to Google Cloud. The code running on Google Cloud is currently dependent on database options only available in Google’s public cloud.  Migrating these databases enables us to run the same code base in Google Cloud and Azure, allowing us to bring recent improvements in our Azure implementation to customers on CDF projects hosted on Google Cloud.   APIs in read-only  modeWe will set the API for the impacted resource types in read-only mode dur

For many operators, power trading remains more of an art than a science that still relies heavily on heuristics and trader intuition. But as markets evolve and deviate from historical patterns, the tools and methods to make smarter trades must adapt in order to maintain or grow profitability and become a sustainable source of competitive advantage.In this webinar, we talk about our experience of integrating data science into forecasting and trading analysis, and how we used Cognite Data Fusion to enable analysts and traders to work iteratively in a data science process across a range of analytical topics within the trading domain. Check it out and leave your comments below! What do you see as the biggest opportunities and challenges within power trading analytics?  

I use RAW as a cache system for sensors that come from BACnet (BAS systems) Since it extremly slow.This have worked for some time now, But now CDF would give me a internal server error.. When this error accure, there is no help what so ever on what the problem is.After debugging, It seems i have reached a size cap on a row (5 MB)  but the table have a size cap of about 5 GB (Unconfirmed)So after i included a size check of my data, I chunk it up and spread it out in the rowsAs fare i can see, this limitiations is not mentioned in the docs so here you go, Liberate your data!

If there is one thing we at Cognite get a lot of questions on, it’s contextualization. Not so much what is contextualization (luckily we are getting past that phase now), but specifically on two subsequent topics:How does your contextualization engine actually work? How does contextualization make use case scaling order of magnitude (or two!) more efficient?In this article, we will address both the above questions. We will also offer an ‘executive summary’ on data contextualization and its role in modern data management towards the end for completeness. Dive in!

A new webinar featuring MHWirth and DNV has just been posted in the Events page.Head over and register now to learn  how MHWirth developed and operationalized their predictive maintenance solution using Industrial DataOps and achieved DNV certification.You can find the link here

Hi Community! I’m Anette Holtedahl, Senior Vice President Industry Applications at Cognite. My team is passionate about creating software that does magic. Our mission is to improve the everyday work for users and equip you with solutions that enable you to make a greater impact: whether that be in the field, the control room, or in a production facility. You are the main character in our mission, and whether you want to work safer, more efficiently, collaborate better, or become more sustainable, we in the application teams work dedicated to enable teams achieve more or avoid human mistakes.  One example of an application that does this, is InField. InField takes advantage of contextualized data from CDF. With iterative product development we could verify how to optimize the functionality to fit with the core users working with operations and maintenance tasks in oil & gas operations. Last module we launched was support for Routine rounds and the adoption has been incredible - also

What are your best practises of sharing?It is proven that when we share, we learn more and grow together!We encourage our partners to share stories, the great stuff you are experiencing with customers out there. @niko (itera), I know you are working on some great  stuff, we are curious to learn more :-)