Community Discussions

When I run the query below, I am getting an error.  If I increase the end time by 1 second I can get a result, but it gives me 2 days and I only want 1.  All my data is consistently daily at midnight, UTC.  Any ideas how to get past this?SELECT  dp.id,  dp.externalId AS `key`,  dp.timestamp,  dp.valueFROM  `_cdf`.`datapoints` dpWHERE  dp.externalId IN (    'f18ae31cbfe544cb7bca08da91e5245a-SuFMiCo0f7c8eca4ce444208853aedd00ded4fb',    '730619216b764ba77bc708da91e5245a-SuFMiCo6bb90df0efbc461d863faedd00df009b'  )  AND dp.timestamp >= TO_TIMESTAMP('2022-09-01T00:00:00Z')  AND dp.timestamp < TO_TIMESTAMP('2022-09-02T00:00:00Z') Gives me a result, but not what I want:  cc @Torgrim Aas  @Sunil Krishnamoorthy 

Sesam.io is a master data hub that simplifies the process of making up-to-date master data available in a data platform architecture.Cognite CDF, well, you all know what is it, right?And we are going to send some data from Sesam to CDF without creating and deploying any connectors, only by using out-of-box functionality provided by Sesam. What we need:Cognite CDF project configured with an Oauth2 identity provider, such as Azure Active directory  Credentials such as client id, secret, oauth token url and scope with access capabilities that allow us to write to Raw tables Provisioned Sesam node with some data.First login into our Sesam instance and create an endpoint system we will use to send data through (I assume you already have some data you want to send to CDF) by choosing “Systems” in left-side menu and then “New system” on the top bar . We will use REST API and push data by making HTTP requests towards CDF. Sesam provides built in REST connector that supports basic and Oauth2 au

I have been looking into many work-orders looking for failures or anomalies, but at this point I’m not sure to take the most of the shared information, as example I’ll take the two following events: The work order 6516382362467974 has the following details available:  Description: Exterior leakage - liquid for the main process Shutdown: Revision (full shutdown) Planned Start: 2014-09-11 I also plotted some of the signals from near the planned start, as shown in the below image:  I have a few questions about the first work order:   It seems that the compressor turned off about two weeks before the planned start time (As the green dotted line shows)   is it safe to assume that is discrepancy is because the two-week delay in sharing the data?  Is there a set of signals that I can look into for this work-order, I’m assuming that since there is a leakage, the time series related to lube oil could be affected, or maybe the pressure of the compressor.   The second work order is 54865032

If you are curious on how to get the most out of your Cognite Data Fusion subscription, you have come to the right place. This is part of a series of posts where we share some of our experience from working with customers in their journey towards an Industrial DataOps organization. We want to share lessons learned, mistakes made, good practices observed, and observations of pitfalls and risks. This is not the absolute truth, but hopefully a way to spark good discussions around an inherently complex topic!To quickly introduce ourselves, we are @Arjo Oosten, Digital Transformation Leader, winter sport addict and passionate about driving hands-on digital growth strategies and value based decision making, and @Karolina Luna, Solution Architect, cat lover, and passionate about the lifecycle perspective of everything (like solutions and data products). To learn more about Cognite Data Fusion, we recommend this post. Different types of cross-functionalIndustrial DataOps, like all other Ops mo

 TL;DRIf Jetfire CLI self contained packages or Jetfire docker image is used in your deployment pipeline. Please refer to the documentation below on how to make the necessary changes to migrate to new Transformations CLI. If not, then no action is required from your end. We have handled it for you! :)) As our services mature, we sometimes need to deprecate the tools and services we made earlier. Last December, we released Transformations API with that came great documentation, support in the Python SDK, and a brand new Python-based Transformations CLI for use in CI CD pipelines.The new Transformations CLI replaces the (old) Jetfire-cli, which will no longer be maintained and will eventually be deprecated at the end of December 2022. The new Transformation CLI supports everything the old Jetfire CLI did and more. It is easier to install, and you can still use your existing Transformations scripts with the new CLI with the legacy mode enabled (Refer migration guide below 👇 ).Like with a

In this conversation I will explain / help how to access the data from the Open Industrial Data Project (OID) CDF Project, through the Cognite Python SDK, with authentication through an interactive login token from Cognite Hub’s Azure Active Directory. The video in the article will step by step show you how you can get access to the data, and also link to the relevant information you will need. Steps:1. Install or update the following python modules/librariespip install cognite-sdkpip install msalpip install pandas2. Download the publicdata.py file and store it in your python environments working directory. this file is prefilled with the OIDC ID’s and CDF project values needed to access the CDF project.3. Open a Jupyter Notebook or python file and import the publicdata.pyfrom publicdata import c4. Access the OID CDF project through the CogniteClient object: c, and when the first method is called on the CogniteClient object it will authenticate and redirect towards the AAD tenant and u

In the TNT application at Statnett we are seeking to collect line and point geospatial features through the FeatureAPI. We have a dataset that has been populated with data about powerlines and substations, and we want to query the client for all of the features in each of these seperately. As there is no list() method in the FeatureAPI, and the search method is limited to 1000 results, we try to do this using the searchStream() method. Our query looks like this:client.geospatial.feature.searchStream('ACLineSegment', { limit: 10000, output: { jsonStreamFormat: 'NEW_LINE_DELIMITED' as const } }); This seems to work fine for the ‘ACLineSegment’ features but when i replace it with ‘Substation’ the call returns an empty string. I’ve tested with the regular search() and this returns features so I would expect the searchStream method to find these as well. Even stranger is that i quickly tested this functionality before the weekend and it seemed to wor

I get the following warning every time I create a CogniteClient: ResourceWarning: unclosed <ssl.SSLSocket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.xx.xx.xx', 49572), raddr=('40.xx.xx.xx', 443)> Is this caused by the connection handling in Cognite SDK? I’m using the following Python modules:cognite-sdk-4.11.0cognite-sdk-core-2.56.1

We proudly announce that Flexible Data Modeling has reached public beta. Flexible data modelling in CDF enables you to model industrial data the way you understand it in a self-documented way so that domain experts can find, recognize and understand the data without a third-party manual.pdf. Your data is stored in one single knowledge graph, and empowers users to search, filter and aggregate data according to their needs through various data models on top of the knowledge graph. Flexible data modeling enables rapid scaling, whether it is building an app, data science model, dashboard, or other use cases, we hope that this capability can be a great toolkit in helping you to express data in the most intuitive way.In the Beta version you can Create and manage data models in Fusion and using a cli tool Ingest data into the data model using CDF transformations Query data using GraphQL with powerful search and filtering in Fusion or your own applications with even more powerful features

Hi there, I’m new to Cognite in general, but one task that my team is looking into is utilizing Cognite’s SVG files in our LCI and Plant Management applications and I have been looking into it somewhat.Not knowing all the details of the SVG rendition capabilities, I do see that our customer ‘Interactive P&IDs’ dataset in CDF has SVGs renditions without the comments that are in the PDF files (where applicable).Would Cognite be able to include the comments - now or in the future?The reason this is relevant for us is that a number of our doc profiles only have PDF files. Typically old documents on old installations.We’re missing native files (DGN, DWG), thus contractors are updating these drawings using comment features such as clouds, text, lines, etc in Adobe. That information seems to disappear in SVG renditions, but it’s quite critical information, especially in Plant Management systems. Thanks for listening to me!Best regards,Erik

OpenID Connect has been enabled on Open Industrial Data for three months.We will be removing the option to use API keys for authentication on November 1st, 2022.If you are still using API keys for authentication, please change the Authenticatioon to openID Connect. Learn how to do that in this article.

Developing, tracking and meeting sustainability goals is becoming increasingly important for companies in the public sphere. Many of them are turning to data-driven solutions to help them monitor, report and reduce their environmental impact. Recent regulations in Europe recognises that Data-driven solutions for GHG emissions reductions are contributing to climate change mitigation (see official site from the European Commission - link) At Cognite, we have encountered many innovative solutions that promote sustainability. Examples include:  Automating the recording and reporting of greenhouse gas emissions in industrial processes. Optimizing energy efficiency at the equipment and system level to minimize scope 2 emissions. Using robotics to detect dangerous leaks. Leveraging operational data to derive the environmental footprint of discrete products. Protecting biodiversity with the automatic detection of birds close to onshore windmills  After analysing dozens of those solutio

I have found a problem reading timeseries from Cognite with Power BI. The first bit of Power Query has a filter to limit the timeseries retrieved to just the ones I want. The second one is the same, but does not have the filter. What I have found is that adding the filter causes duplicate rows in the table. Once the timeseries rows are duplicated, the timeseries aggregation values are also duplicated. I know that the first query is not the most performant option, but it should still have the correct answer.let Source = Cognite.Contents( #"cogniteContentsParameters", #"cdfEnvironment"), Timeseries_table = Source{[Name="Timeseries",Signature="table"]}[Data], #"Filtered Rows" = Table.SelectRows(Timeseries_table, each ([IsString] = false)), #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"MetaData", "SecurityCategories", "IsStep", "DataSetId", "CreatedTime", "LastUpdatedTime", "Asset", "Latest", "DataPoints", "StringPoints", "Aggregate"}), #"Filtered Rows1" = Tabl

Hi! After switching from experimental to cognite-sdk 3.9.0 I struggle with deployment. 1: Using handleIf i try to deploy using the handle, then the requirements are not handled. The function deploys, but failing at the imports. How should requiremenents be used when deploying like this? Imports are inside handle():def handle(client, data, secrets): from googleapiclient.discovery import build from googleapiclient.errors import HttpErrorCreating function like this:cognite_client.functions.create( name=function_name, external_id=function_ext_id, function_handle=handle)2 - Using a folder.Then I get this error: [WinError 267] The directory name is invalid: 'C:\\Users\\hknap\\AppData\\Local\\Temp\\tmptzkywbdl\\function.zip'If I debug, the folder “tmptzkywbdl” is created, so there should be no permission issues.  3- Manually uploading the zip fileThis works (including the requirements.txt), thanks for adding this possibility. It is just a little extra hassle ;)  would prefer if

Hi!Almost three years ago we made a feature request regarding search. Specifically we want to:Have support for paging. Do grouping on metadata-fields. Define which fields yields relevance to the search-results. Inclusive search-terms using AND instead of OR.I do understand that satisfying the search-requirements of all your users is a daunting task, but perhaps it would be easier to let us do the job instead? I believe it would be very beneficial if we were able to access ElasticSearch directly in some manner.And please keep in mind that the data itself defines relevancy. No two customers are alike and therefore a good generic solution likely doesn’t exist. @Knut Vidvei @Andreea Pastinaru 

Schedule a personalized demo to learn how Cognite Data Fusion™ generates fast, scalable value from your data, enabling better decision making about maintenance, production and safety.In the demo, we will:Understand your priorities, initiatives and challenges that you are looking to solve Introduce Cognite and share examples of use cases we deliver for our customers today Identify areas where Cognite Data Fusion can help your organizationRequest demo with one of our product experts 

If you are curious on how to get the most out of your Cognite Data Fusion subscription, you have come to the right place. This is part of a series of posts where we share some of our experience from working with customers in their journey towards an Industrial DataOps organization. We want to share lessons learned, mistakes made, good practices observed, and observations of pitfalls and risks. This is not the absolute truth, but hopefully a way to spark good discussions around an inherently complex topic!To quickly introduce ourselves, we are @Arjo Oosten, Digital Transformation Leader, winter sport addict and passionate about driving hands-on digital growth strategies and value based decision making, and @Karolina Luna, Solution Architect, cat lover, and passionate about the lifecycle perspective of everything (like solutions and data products). To learn more about Cognite Data Fusion, we recommend this post.Planning your solutions and Industrial DataOps with Cognite Data FusionTo be

Hello Charts Early Adopter Community,I know it’s been a bit quiet in this group lately, but rest assured, the team and I have been hard at work. We have just released several new features and fixes to production. You can scroll down to read about each of them in detail.Please do leave comments below with questions and feedback!What’s coming next?We’re in the middle of our development cycle in preparation for the next major release of Cognite Data Fusion in December 2022. There are the two major features our team is working on that you can look forward to having available:Moving Charts (charts.cogniteapp.com) into Cognite Data Fusion (fusion.cognite.com) We’ve received plenty of feedback that our product experience can and need to be more tightly integrated and we’ve heard you loud and clear. As you all know, Charts is currently available on charts.cogniteapp.com, which is separate from Cognite Data Fusion (fusion.cognite.com). Despite starting out on its standalone URL, Charts has alw

In many cases in diagrams including P&ID, there are shortcuts to represent tags, for instance:F2014A/B/C for F2014A , F2014B and F2014C.The contextualization engine in some cases will pick one of the 3 or none at all.Is there a way for the engine to detect trailing slashes and match to multiple tags?See example of patterns below:Example of patterns 

I'm working on identifying the falling and rising edges of the VAL_23-KA-9101-M01:HSI.StatusMotorOn signal. For that and shifting the TS and rest the values, if I get -1 is a falling edge and for 1 it is a rising edge.  For the first row it works fine:  But I’m also getting a lot of values different than 1 or -1. Especially for more recent years. I guess that the problem is at the aggregating step, since I downloaded the data at a 1m frequency I get not int values for some records. Now, I was trying to download this TS at a 1s frequency, but I get a different number of records depending on the year.  E.G. for 2014 I get 34 records for a 10 days time windows with a granularity 1m.  If I do the same but for 2020 I get 1411, much more than for 2014, but still a tenth of the expected count.  If I change the granularity to 1s, I still get the same number of records, but with more precision in the time stamp:   Considering this, is there a recommended granularity to download the data and min

I am listing time series for a given asset, and get a lot of results. I need to filter based on the time series external ID, but the only option is by using the “external_id_prefix” argument to the list function. So I build up the prefix left to right. Somewhere in the external id is a parameter that I do not care about, and after comes a new parameter that I want a particular value of. Concrete example of external IDs:IAA_Miros_Weather_Data_WIA_008IAA_Miros_Weather_Data_WIB_008IAA_Miros_Weather_Data_WIC_008IAA_Miros_Weather_Data_WID_008IAA_Miros_Weather_Data_WIE_008I am interested in only getting time series with external IDs that contain “_WI” and that end with “008”. Is there a way to list time series with a wildcard? Something like this:client.time_series.list( asset_ids=[my_asset_id], limit=None, external_id_prefix="IAA_Miros_Weather_Data_WI*_008", partitions=4)I could of course obtain the relevant time series by filtering after the fact. Something like thisresult = cl

Do you know what courses are available for you to explore the data from the Open Industrial Data project?Cognite Academy creates e-learning courses with an emphasis on providing hands-on experiences so you can learn to use Cognite Data Fusion through interacting with real industrial data. In most of our courses, the training data we use comes from the Open Industrial Data (OID) project, a live stream of industrial data from the Valhall oil platform. While the data is from the oil and gas industry, it is relevant for all asset-heavy industries as it provides insights into dynamic industrial processes. In this article, I will share a series of CDF & Power BI courses where you can explore the Open Industrial Data and solve a simple use case in Power BI: Introduction to CDF & Power BI  In this course, you will connect Power BI to CDF, and retrieve the Open Industrial Data to learn how filtering and aggregation work with the connector in Power BI.  CDF & Power BI: Solving

Are you a Power BI user who wants to gain hands-on experience with Cognite Data Fusion? Are you curious to learn how to handle a large set of CDF data in Power BI?The Cognite Power BI connector lets you use a CDF project as a data source in Power BI Desktop to query, transform and visualize data, share insights across your organization, or embed dashboards in your app or website. When working with the Cognite Power BI connector, it is important to know how to refresh the data and apply best practices to deliver accurate results. The newest course from Cognite Academy introduces you to implementing incremental refresh - one of the best practices to get the most out of the Cognite Power BI connector. Our instructors first explain when to use incremental refresh, then show you how to set it up in four steps. In this course, the training data comes from the Open Industrial Data (OID) project, a live stream of industrial data from the Valhall oil platform.After completing this course, you’l

Hi Team I am trying to deploy the code into new env facing issue because of black --check.