Community Discussions

Hi everyone,We in C4IR Ocean are starting a series where we are challenging the community to model a certain problem in CDF. The aim with this series is to facilitate discussion and invite community members to share interesting solutions and techniques.The first challenge focuses on Open LineageOpenLineage is an Open standard for metadata and lineage collection designed to instrument jobs as they are running. It defines a generic model of run, job, and dataset entities identified using consistent naming strategies. The core lineage model is extensible by defining specific facets to enrich those entities.In

One of the main objectives of the first deployment with an inspection robot is getting your organization ready for robotics. A properly scoped deployment will contribute to engaging the field workers and provide insights on what it takes to keep robots operational on a day-to-day basis. This will lay the foundation for high return on investment for all the following deployments. If the scoping fails, there is a risk of losing trust in robots and gaining less willingness to change in your organization. But fear not. In this post, we’re summarizing our best practices when assisting our customers to scope out their first deployment of an inspection robot.  

In the CDF docs I see there is a PI and OPCUA connector. Many industry 4.0 community members see MQTT rather than OPCUA as the future of Industrial IoT cloud based communication, as it is lightweight, report by exception, client driven, etc…If I want to onboard a facility that has an IoT MQTT gateway, how does Cognite recommend we integrate? I have used Azure IoTHub and Azure Functions to connect and transform the data in the past. Do you have a reference architecture for this? Can we connect to the gateway directly with a CDF connector, or do you recommend using Azure IoTHub and Functions (or similar GCP/AWS)?Also Azure has some neat tools like IoT Device Provisioning Services, be nice to know a POV on if/how to utilize those. Thanks! 

The majority of the hydropower fleet is pretty old. It’s not uncommon to see a 100-year-old power plant still in operation. While it may be a great piece of engineering, how can you access operational data on your laptop or smartphone? After all, you may not want to travel every day to very remote locations. At Ringerikskraft’s Hønefoss II power station in Norway, we installed a set of cameras to passively record alarm events, read gauges, and stream this information to Cognite Data Fusion. It is all connected, so if an alarm goes off, I get an SMS notification and can quickly look up what is happening. Pretty cool, or what? I would like to hear your thoughts. How have you tackled the challenge of digitalizing aging infrastructure?Read the full

Hi there,I am looking for resources on CDF templates API.So far I the best introduction I have found is the Python SDK documentation, as well as the information at docs.cognite.com which focuses on using templates in fusion.Can you provide more information about how to use the templates API?

I am trying to stream data from CDF to Azure Event Hub with the Python SDK and cannot find anything related to streaming datasets. Only option so far (as I know) is dps = c.datapoints.retrieve_latest(id=184691546499795)which would need a trigger of some sort to keep running. Data from CDF are so called timeseries from different types of sensors. Is there any documentation on Streaming Data for CDF that I could look at or is Streaming really supported? 

Hi, When developing with the Cognite python SDK, a common restriction is  the API imposed restrictions on queries. Quering time-series, for example, is restricted by 100 asset-ids.  I believe the Python SDK should recognize these constraint violations and batch / concurrently dispatch requests in chunks that do not violate constraints. Optionally with a performance-warning to the developer.Is this sane, or do you think that the developer/customer should maintain a wrapper on the SDK for batching each endpoint (as we’ve currently done at Statnett)?

 If a user includes https:// or http:// in the Override Azure Tenant field the url to Azure AD will not include a correct tenant and login is prevented. This has caused login issues for at least a few users.  The request is simply that the form is modified to strip the protocol scheme, ie removing http:// or https:// automatically, if present.  

Hey, In our current workflow we’re expanding the use of Functions as a tool. We’re somewhat hampered by how “clumsy” it is to bundle and version control proprietary dependencies. Is there something in the pipe to address this issue? Kind regards,Robert

Hey!In our tooling for the power-analyst we make computations using the SyntheticTimeseries API. Results from these are used in subsequent analysis, where we have identified a problem for us.    When a synthetic timeseries is computed with a specified aggregate and resolution, that specification is returned  irrespective of there being data on the originating timeseries. In our case, we compute aggregates over long stretches of time which results in situations like the image below: In the figure, the opaque line is the comptued by addition of the two other signals. Spanning over roughly a year, the period in the middle has a linear rise in the SyntheticTimeseries while the originals are empty. These values are of course meaningless and should be omitted in the successive steps of the analysis. Have you con

 McKinsey Quarterly did post an interesting set of articles recently, regarding the transition and change management when looking into digital services. I believe some of the topic and views could be interesting to reflect upon. Go here to find out : https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/five-fifty-so-you-want-to-be-a-software-company?cid=other-eml-alt-mip-mck&hdpid=034603b3-3fba-4313-a963-553dca543b0f&hctky=12386931&hlkid=8635b3c5c91d439a8a47f0bf353a4968  

How a DataOps platform can boost improvement in all parts of the business! And this time is focusing on what is most important - HSE!https://www.pgs.com/investor-relations/ir-news-stock-announcements/pgs-digitalization-initiative-improves-crew-safety/https://www.cognite.com/customers_stories/dataops-in-action-improving-vessel-crew-safety One more big shout out to @Cerys James and @Sverre Olsen ! Congratulations for another amazing use case!

Hey! I’ve a problem with filling a gap from a source to a timeseries in CDF.  Problem descriptionWe’re filling a hole in a time-series from time A to  B. There are some datapoints on the edges of the interval in CDF.Data is extracted from the source, and in python prepped for the datapoints API as a list-of-tuples payload. For an arbitrary period I extract 2976 datapoints which I upload to CDF. Subsequently, I query the time-series for the same period of time and recieve 2928 datapoints. There are no NAN values in the input for either the date-time or value. The data is also hourly, and so I’m wary of it just being an edge effect of poor timestamp specifications for the retrieval. What other PEBCAK things have I missed? Simplified example included below:payload>> [{'externalId': 'ts_externalid', 'datapoints': [...]}]payload[0]["datapoints"][10]>> (16

 It's been a while, but as promised, here is the follow-up post to my previous article. If you haven't read the previous article, you can find it here.In the first part, I shared the basic concepts about data integration. Today I will continue on the same theme, but the focus will be on latency and frequency. They are related concepts, but not necessarily the same. Why latency and frequency? They are crucial to defining your data pipeline, and they influence which Azure resources you use to cover your needs.What is data latency and frequency? Data latency is how fast/slow data can be retrieved or stored. Low latency means that the data is available in rea

Hi, my name is Kine Årdal, and I have the privilege of being the Customer success manager for Neptune Energy. I’d like to share experiences from our latest project, and hoping to get feedback and hear about similar experiences from you in the community. The project kicked off in April, as a12-weeks proof of concept, where we enabled a collaborative environment with seamless data flow of key well data for well planning.Current workflow: In the current well planning workflow, it is challenging to share data efficiently and reliably across disciplines and collaboration partners. There is no central storage of drilling and well data ready for a digital way of working, and data is often stored in silos. In addition, the collaboration with vendors often relies on manual exports/imports, sharing data on e-mails/ftp, which leads to e.g. unnecessa

We are looking into cloud-optimized storage formats such as GeoTIFF, Parquet, etc. One of the things we are trying to determine is whether we could fully utilize such cloud optimized formats with CDF Files.Will the download-link returned by CDF Files allow us to do seek-operations and only download parts of the file?

Hi. I’m currently working on an application where we need to check if a node (PLC/PC) have updated any timeseries within x amounts of hours. The way we do it now is to get all timeseries recorded values in a time range, and check if any timeseries have any values. If there is an value on one timeserie, then we consider the node alive. This can be time consuming, since each node can several hundreds/thousands of timeseries. So my question is if there is a way to get the latest recorded value in a collection of timeseries within a timerange? Or just a recorded value in a timerange for a collection of timeseries. In Cognite, we have an asset hierarchy, which preferably would look like this:RigNode 1 TimeSerie1 TimeSerie2 ... Node 2 TimeSerie1 TimeSerie2 ... Node ... TimeSerie1 TimeSerie

Hi all,take a look at https://properate.com/, the landing page for Properate. This is the solution Energima have built on CDF. Remember to watch the landing page embedded video :-)Energima is taking full advantage of several CDF services and Properate includes a stack of cool features!  

The happiest moment in a CSM day/week/month/ YEAR is when we see our customers sharing success cases! Thank you PGS for allowing us to be part of your digital journey! Contratualizations for another fantastic achievement @Cerys James @Sverre Olsen !!