Community Discussions

In this conversation I will explain / help how to access the data from the Open Industrial Data Project (OID) CDF Project, through the Cognite Python SDK, with authentication through an interactive login token from Cognite Hub’s Azure Active Directory. The video in the article will step by step show you how you can get access to the data, and also link to the relevant information you will need. Steps:1. Install or update the following python modules/librariespip install cognite-sdkpip install msalpip install pandas2. Download the publicdata.py file and store it in your python environments working directory. this file is prefilled with the OIDC ID’s and CDF project values needed to access the CDF project.3. Open a Jupyter Notebook or python file and import the publicdata.pyfrom publicdata import c4. Access the OID CDF project through the CogniteClient object: c, and when the first method is called on the CogniteClient object it will authenticate and redirect towards the AAD tenant and u

In the TNT application at Statnett we are seeking to collect line and point geospatial features through the FeatureAPI. We have a dataset that has been populated with data about powerlines and substations, and we want to query the client for all of the features in each of these seperately. As there is no list() method in the FeatureAPI, and the search method is limited to 1000 results, we try to do this using the searchStream() method. Our query looks like this:client.geospatial.feature.searchStream('ACLineSegment', { limit: 10000, output: { jsonStreamFormat: 'NEW_LINE_DELIMITED' as const } }); This seems to work fine for the ‘ACLineSegment’ features but when i replace it with ‘Substation’ the call returns an empty string. I’ve tested with the regular search() and this returns features so I would expect the searchStream method to find these as well. Even stranger is that i quickly tested this functionality before the weekend and it seemed to wor

I get the following warning every time I create a CogniteClient: ResourceWarning: unclosed <ssl.SSLSocket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.xx.xx.xx', 49572), raddr=('40.xx.xx.xx', 443)> Is this caused by the connection handling in Cognite SDK? I’m using the following Python modules:cognite-sdk-4.11.0cognite-sdk-core-2.56.1

We proudly announce that Flexible Data Modeling has reached public beta. Flexible data modelling in CDF enables you to model industrial data the way you understand it in a self-documented way so that domain experts can find, recognize and understand the data without a third-party manual.pdf. Your data is stored in one single knowledge graph, and empowers users to search, filter and aggregate data according to their needs through various data models on top of the knowledge graph. Flexible data modeling enables rapid scaling, whether it is building an app, data science model, dashboard, or other use cases, we hope that this capability can be a great toolkit in helping you to express data in the most intuitive way.In the Beta version you can Create and manage data models in Fusion and using a cli tool Ingest data into the data model using CDF transformations Query data using GraphQL with powerful search and filtering in Fusion or your own applications with even more powerful features

Hi there, I’m new to Cognite in general, but one task that my team is looking into is utilizing Cognite’s SVG files in our LCI and Plant Management applications and I have been looking into it somewhat.Not knowing all the details of the SVG rendition capabilities, I do see that our customer ‘Interactive P&IDs’ dataset in CDF has SVGs renditions without the comments that are in the PDF files (where applicable).Would Cognite be able to include the comments - now or in the future?The reason this is relevant for us is that a number of our doc profiles only have PDF files. Typically old documents on old installations.We’re missing native files (DGN, DWG), thus contractors are updating these drawings using comment features such as clouds, text, lines, etc in Adobe. That information seems to disappear in SVG renditions, but it’s quite critical information, especially in Plant Management systems. Thanks for listening to me!Best regards,Erik

OpenID Connect has been enabled on Open Industrial Data for three months.We will be removing the option to use API keys for authentication on November 1st, 2022.If you are still using API keys for authentication, please change the Authenticatioon to openID Connect. Learn how to do that in this article.

Developing, tracking and meeting sustainability goals is becoming increasingly important for companies in the public sphere. Many of them are turning to data-driven solutions to help them monitor, report and reduce their environmental impact. Recent regulations in Europe recognises that Data-driven solutions for GHG emissions reductions are contributing to climate change mitigation (see official site from the European Commission - link) At Cognite, we have encountered many innovative solutions that promote sustainability. Examples include:  Automating the recording and reporting of greenhouse gas emissions in industrial processes. Optimizing energy efficiency at the equipment and system level to minimize scope 2 emissions. Using robotics to detect dangerous leaks. Leveraging operational data to derive the environmental footprint of discrete products. Protecting biodiversity with the automatic detection of birds close to onshore windmills  After analysing dozens of those solutio

I have found a problem reading timeseries from Cognite with Power BI. The first bit of Power Query has a filter to limit the timeseries retrieved to just the ones I want. The second one is the same, but does not have the filter. What I have found is that adding the filter causes duplicate rows in the table. Once the timeseries rows are duplicated, the timeseries aggregation values are also duplicated. I know that the first query is not the most performant option, but it should still have the correct answer.let Source = Cognite.Contents( #"cogniteContentsParameters", #"cdfEnvironment"), Timeseries_table = Source{[Name="Timeseries",Signature="table"]}[Data], #"Filtered Rows" = Table.SelectRows(Timeseries_table, each ([IsString] = false)), #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"MetaData", "SecurityCategories", "IsStep", "DataSetId", "CreatedTime", "LastUpdatedTime", "Asset", "Latest", "DataPoints", "StringPoints", "Aggregate"}), #"Filtered Rows1" = Tabl

Hi! After switching from experimental to cognite-sdk 3.9.0 I struggle with deployment. 1: Using handleIf i try to deploy using the handle, then the requirements are not handled. The function deploys, but failing at the imports. How should requiremenents be used when deploying like this? Imports are inside handle():def handle(client, data, secrets): from googleapiclient.discovery import build from googleapiclient.errors import HttpErrorCreating function like this:cognite_client.functions.create( name=function_name, external_id=function_ext_id, function_handle=handle)2 - Using a folder.Then I get this error: [WinError 267] The directory name is invalid: 'C:\\Users\\hknap\\AppData\\Local\\Temp\\tmptzkywbdl\\function.zip'If I debug, the folder “tmptzkywbdl” is created, so there should be no permission issues.  3- Manually uploading the zip fileThis works (including the requirements.txt), thanks for adding this possibility. It is just a little extra hassle ;)  would prefer if

Hi!Almost three years ago we made a feature request regarding search. Specifically we want to:Have support for paging. Do grouping on metadata-fields. Define which fields yields relevance to the search-results. Inclusive search-terms using AND instead of OR.I do understand that satisfying the search-requirements of all your users is a daunting task, but perhaps it would be easier to let us do the job instead? I believe it would be very beneficial if we were able to access ElasticSearch directly in some manner.And please keep in mind that the data itself defines relevancy. No two customers are alike and therefore a good generic solution likely doesn’t exist. @Knut Vidvei @Andreea Pastinaru 

Schedule a personalized demo to learn how Cognite Data Fusion™ generates fast, scalable value from your data, enabling better decision making about maintenance, production and safety.In the demo, we will:Understand your priorities, initiatives and challenges that you are looking to solve Introduce Cognite and share examples of use cases we deliver for our customers today Identify areas where Cognite Data Fusion can help your organizationRequest demo with one of our product experts 

If you are curious on how to get the most out of your Cognite Data Fusion subscription, you have come to the right place. This is part of a series of posts where we share some of our experience from working with customers in their journey towards an Industrial DataOps organization. We want to share lessons learned, mistakes made, good practices observed, and observations of pitfalls and risks. This is not the absolute truth, but hopefully a way to spark good discussions around an inherently complex topic!To quickly introduce ourselves, we are @Arjo Oosten, Digital Transformation Leader, winter sport addict and passionate about driving hands-on digital growth strategies and value based decision making, and @Karolina Luna, Solution Architect, cat lover, and passionate about the lifecycle perspective of everything (like solutions and data products). To learn more about Cognite Data Fusion, we recommend this post.Planning your solutions and Industrial DataOps with Cognite Data FusionTo be

Hello Charts Early Adopter Community,I know it’s been a bit quiet in this group lately, but rest assured, the team and I have been hard at work. We have just released several new features and fixes to production. You can scroll down to read about each of them in detail.Please do leave comments below with questions and feedback!What’s coming next?We’re in the middle of our development cycle in preparation for the next major release of Cognite Data Fusion in December 2022. There are the two major features our team is working on that you can look forward to having available:Moving Charts (charts.cogniteapp.com) into Cognite Data Fusion (fusion.cognite.com) We’ve received plenty of feedback that our product experience can and need to be more tightly integrated and we’ve heard you loud and clear. As you all know, Charts is currently available on charts.cogniteapp.com, which is separate from Cognite Data Fusion (fusion.cognite.com). Despite starting out on its standalone URL, Charts has alw

In many cases in diagrams including P&ID, there are shortcuts to represent tags, for instance:F2014A/B/C for F2014A , F2014B and F2014C.The contextualization engine in some cases will pick one of the 3 or none at all.Is there a way for the engine to detect trailing slashes and match to multiple tags?See example of patterns below:Example of patterns 

I'm working on identifying the falling and rising edges of the VAL_23-KA-9101-M01:HSI.StatusMotorOn signal. For that and shifting the TS and rest the values, if I get -1 is a falling edge and for 1 it is a rising edge.  For the first row it works fine:  But I’m also getting a lot of values different than 1 or -1. Especially for more recent years. I guess that the problem is at the aggregating step, since I downloaded the data at a 1m frequency I get not int values for some records. Now, I was trying to download this TS at a 1s frequency, but I get a different number of records depending on the year.  E.G. for 2014 I get 34 records for a 10 days time windows with a granularity 1m.  If I do the same but for 2020 I get 1411, much more than for 2014, but still a tenth of the expected count.  If I change the granularity to 1s, I still get the same number of records, but with more precision in the time stamp:   Considering this, is there a recommended granularity to download the data and min

I am listing time series for a given asset, and get a lot of results. I need to filter based on the time series external ID, but the only option is by using the “external_id_prefix” argument to the list function. So I build up the prefix left to right. Somewhere in the external id is a parameter that I do not care about, and after comes a new parameter that I want a particular value of. Concrete example of external IDs:IAA_Miros_Weather_Data_WIA_008IAA_Miros_Weather_Data_WIB_008IAA_Miros_Weather_Data_WIC_008IAA_Miros_Weather_Data_WID_008IAA_Miros_Weather_Data_WIE_008I am interested in only getting time series with external IDs that contain “_WI” and that end with “008”. Is there a way to list time series with a wildcard? Something like this:client.time_series.list( asset_ids=[my_asset_id], limit=None, external_id_prefix="IAA_Miros_Weather_Data_WI*_008", partitions=4)I could of course obtain the relevant time series by filtering after the fact. Something like thisresult = cl

Do you know what courses are available for you to explore the data from the Open Industrial Data project?Cognite Academy creates e-learning courses with an emphasis on providing hands-on experiences so you can learn to use Cognite Data Fusion through interacting with real industrial data. In most of our courses, the training data we use comes from the Open Industrial Data (OID) project, a live stream of industrial data from the Valhall oil platform. While the data is from the oil and gas industry, it is relevant for all asset-heavy industries as it provides insights into dynamic industrial processes. In this article, I will share a series of CDF & Power BI courses where you can explore the Open Industrial Data and solve a simple use case in Power BI: Introduction to CDF & Power BI  In this course, you will connect Power BI to CDF, and retrieve the Open Industrial Data to learn how filtering and aggregation work with the connector in Power BI.  CDF & Power BI: Solving

Are you a Power BI user who wants to gain hands-on experience with Cognite Data Fusion? Are you curious to learn how to handle a large set of CDF data in Power BI?The Cognite Power BI connector lets you use a CDF project as a data source in Power BI Desktop to query, transform and visualize data, share insights across your organization, or embed dashboards in your app or website. When working with the Cognite Power BI connector, it is important to know how to refresh the data and apply best practices to deliver accurate results. The newest course from Cognite Academy introduces you to implementing incremental refresh - one of the best practices to get the most out of the Cognite Power BI connector. Our instructors first explain when to use incremental refresh, then show you how to set it up in four steps. In this course, the training data comes from the Open Industrial Data (OID) project, a live stream of industrial data from the Valhall oil platform.After completing this course, you’l

Hi Team I am trying to deploy the code into new env facing issue because of black --check.

HiI have retrieved timeseries from an asset in PowerBi. That asset is linked through a relationship column to a group of assets. However the asset and the group it belongs to are connected trough 4 steps:groupwanted <- belongsTo - subgroup<- belongsTo - type of group <- connectsTo - group of a few timeseries assets<- belongsTo - timeseries asset Im wondering how to link the relationships in power bi, and how to orient to the right place. 

Hi Team I am trying to deploy ML Model into the AIR but Creating Env Variable Step is showing the error. Also attached image consisting issue details. Regards.

On serveral occations we have encountered limitations in metadata key length, most frequently when flattening JSON-formatted strings from our event stream data source systems. When the source system presents nested structure of as many as 4 levels we frequently encounter metadata keys that require more than 128 bytes. Up to now we have “solved” the issue by abbreviating the metadatakeys at the price of higher maintenance cost of the code and more importantly, that end-users get the perception that we have transformed the data or even dont understand what it represents. We now consider moving towards a solution where we simply put the entire JSON-formatted string into one single metadata value field, and leave to front-end teams and end users to flatten the structure. We have a similar issue with max number of metadata keys for timesseries (16). Question 1) Could you please suggest other, better options for handling these metadata limitations? Question 2) Will templates come to the resc

Hi everyone, I tried to upload my own dataset(time-series data) into CDF, but I always get error: Request with id 87b235ea-78dd-963a-b343-ff58a0ee084d to https://westeurope-1.cognitedata.com/api/v1/projects/learn/timeseries/data failed with status 400: Timestamp is too high, must be at most 2556143999999 which is 31-12-2050 23:59:59 GMT.I followed with this link:https://learn.cognite.com/path/cognite-data-fusion-fundamentals/working-with-cdf-integrate. I created an assest, 3 time-series. What I want to do is to upload my data to the 3 time-series, and show the variation of my data in CDF. I change the time_stamp of my data into the format of 2022-02-10T17:00:00.000Z, I am not sure if this is necessary but when I preview my datapoints in RAW explorer and CDF automatically change the this format into for example, 164998440000, so I am wondering if this is why I have this error? I also want to ask what is the meaning of span in IFSDB.sensors? It means this time-series is 100 days? or 100

Screenshot for reference.