Product User Community

If you are curious on how to get the most out of your Cognite Data Fusion subscription, you have come to the right place. This is part of a series of posts where we share some of our experience from working with customers in their journey towards an Industrial DataOps organization. We want to share lessons learned, mistakes made, good practices observed, and observations of pitfalls and risks. This is not the absolute truth, but hopefully a way to spark good discussions around an inherently complex topic!To quickly introduce ourselves, we are @Arjo Oosten, Digital Transformation Leader, winter sport addict and passionate about driving hands-on digital growth strategies and value based decision making, and @Karolina Luna, Solution Architect, cat lover, and passionate about the lifecycle perspective of everything (like solutions and data products). To learn more about Cognite Data Fusion, we recommend this post. Outsourcing FTW?As many companies have found, it can be hard to attract the

In grafana, we want to display total of all timeseries data-points for all assets connected to particular asset for which we are passing query like this:ts{assetSubtreeIds=[{id=$Org4}], name='oil-production-rate'}where value of $Org1 is coming from dashboard variable.and since cognite grafana plugin does not handle cursor api responses and synthetic timeseries api have some limitation in returning 10k records only in a request, to reduce number of data-points, we are trying to use aggregation: ‘sum’ and granularity: ‘30d’ since there could be more than 10k data-points. But its is throwing error saying “sum” is not valid aggregation type of synthetic timeseries. Any reason why “sum” aggregation is not supported for synthetic timeseries.(screenshot attached)

Hi, I am currently doing my Data Engineer Basics - “Learn to Use the Cognite Python SDK “ Course. while executing the notebooks, getting below error with the first “List” notebook even after successful Cognite client authentication.CogniteAPIError: Resource not found. This may also be due to insufficient access rights. | code: 403 | X-Request-ID: 52b3d730-caea-9d32-bfc4-649041e43c3e Note: I am not using Poetry as I am not getting option to change Kernel in my Jupyter.

If you are curious on how to get the most out of your Cognite Data Fusion subscription, you have come to the right place. This is part of a series of posts where we share some of our experience from working with customers in their journey towards an Industrial DataOps organization. We want to share lessons learned, mistakes made, good practices observed, and observations of pitfalls and risks. This is not the absolute truth, but hopefully a way to spark good discussions around an inherently complex topic!To quickly introduce ourselves, we are @Arjo Oosten , Digital Transformation Leader, wintersport addict and passionate about driving hands-on digital growth strategies and value based decision making, and @Karolina Luna , Solution Architect, cat lover, and passionate about the lifecycle perspective of everything (like solutions and data products). To learn more about Cognite Data Fusion, we recommend this post. Knowing which roles you need for your Industrial DataOps organization, as c

Data operations (DataOps) is essential to providing consumers with business-ready, trusted, high-quality data. But when faced with a somewhat different data source, data type, data quality, and data consumer landscape, what are the defining factors that will significantly catalyze DataOps adoption in industrial companies? Join our upcoming webinar with Forrester, and be a part of the discussion, as we unpack the technology behind the DataOps practice. Dive into the topic of data operations for industry and hear leading analysts’ take on:-What is DataOps today? How has it evolved over the years?-How does DataOps adoption affect industrial companies?-What are the challenges when adopting DataOps?-Where do DataOps and MLOps come together? Register here

When I run the query below, I am getting an error.  If I increase the end time by 1 second I can get a result, but it gives me 2 days and I only want 1.  All my data is consistently daily at midnight, UTC.  Any ideas how to get past this?SELECT  dp.id,  dp.externalId AS `key`,  dp.timestamp,  dp.valueFROM  `_cdf`.`datapoints` dpWHERE  dp.externalId IN (    'f18ae31cbfe544cb7bca08da91e5245a-SuFMiCo0f7c8eca4ce444208853aedd00ded4fb',    '730619216b764ba77bc708da91e5245a-SuFMiCo6bb90df0efbc461d863faedd00df009b'  )  AND dp.timestamp >= TO_TIMESTAMP('2022-09-01T00:00:00Z')  AND dp.timestamp < TO_TIMESTAMP('2022-09-02T00:00:00Z') Gives me a result, but not what I want:  cc @Torgrim Aas  @Sunil Krishnamoorthy 

Sesam.io is a master data hub that simplifies the process of making up-to-date master data available in a data platform architecture.Cognite CDF, well, you all know what is it, right?And we are going to send some data from Sesam to CDF without creating and deploying any connectors, only by using out-of-box functionality provided by Sesam. What we need:Cognite CDF project configured with an Oauth2 identity provider, such as Azure Active directory  Credentials such as client id, secret, oauth token url and scope with access capabilities that allow us to write to Raw tables Provisioned Sesam node with some data.First login into our Sesam instance and create an endpoint system we will use to send data through (I assume you already have some data you want to send to CDF) by choosing “Systems” in left-side menu and then “New system” on the top bar . We will use REST API and push data by making HTTP requests towards CDF. Sesam provides built in REST connector that supports basic and Oauth2 au

If you are curious on how to get the most out of your Cognite Data Fusion subscription, you have come to the right place. This is part of a series of posts where we share some of our experience from working with customers in their journey towards an Industrial DataOps organization. We want to share lessons learned, mistakes made, good practices observed, and observations of pitfalls and risks. This is not the absolute truth, but hopefully a way to spark good discussions around an inherently complex topic!To quickly introduce ourselves, we are @Arjo Oosten, Digital Transformation Leader, winter sport addict and passionate about driving hands-on digital growth strategies and value based decision making, and @Karolina Luna, Solution Architect, cat lover, and passionate about the lifecycle perspective of everything (like solutions and data products). To learn more about Cognite Data Fusion, we recommend this post. Different types of cross-functionalIndustrial DataOps, like all other Ops mo

 TL;DRIf Jetfire CLI self contained packages or Jetfire docker image is used in your deployment pipeline. Please refer to the documentation below on how to make the necessary changes to migrate to new Transformations CLI. If not, then no action is required from your end. We have handled it for you! :)) As our services mature, we sometimes need to deprecate the tools and services we made earlier. Last December, we released Transformations API with that came great documentation, support in the Python SDK, and a brand new Python-based Transformations CLI for use in CI CD pipelines.The new Transformations CLI replaces the (old) Jetfire-cli, which will no longer be maintained and will eventually be deprecated at the end of December 2022. The new Transformation CLI supports everything the old Jetfire CLI did and more. It is easier to install, and you can still use your existing Transformations scripts with the new CLI with the legacy mode enabled (Refer migration guide below 👇 ).Like with a

In the TNT application at Statnett we are seeking to collect line and point geospatial features through the FeatureAPI. We have a dataset that has been populated with data about powerlines and substations, and we want to query the client for all of the features in each of these seperately. As there is no list() method in the FeatureAPI, and the search method is limited to 1000 results, we try to do this using the searchStream() method. Our query looks like this:client.geospatial.feature.searchStream('ACLineSegment', { limit: 10000, output: { jsonStreamFormat: 'NEW_LINE_DELIMITED' as const } }); This seems to work fine for the ‘ACLineSegment’ features but when i replace it with ‘Substation’ the call returns an empty string. I’ve tested with the regular search() and this returns features so I would expect the searchStream method to find these as well. Even stranger is that i quickly tested this functionality before the weekend and it seemed to wor

I get the following warning every time I create a CogniteClient: ResourceWarning: unclosed <ssl.SSLSocket fd=5, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('10.xx.xx.xx', 49572), raddr=('40.xx.xx.xx', 443)> Is this caused by the connection handling in Cognite SDK? I’m using the following Python modules:cognite-sdk-4.11.0cognite-sdk-core-2.56.1

We proudly announce that Flexible Data Modeling has reached public beta. Flexible data modelling in CDF enables you to model industrial data the way you understand it in a self-documented way so that domain experts can find, recognize and understand the data without a third-party manual.pdf. Your data is stored in one single knowledge graph, and empowers users to search, filter and aggregate data according to their needs through various data models on top of the knowledge graph. Flexible data modeling enables rapid scaling, whether it is building an app, data science model, dashboard, or other use cases, we hope that this capability can be a great toolkit in helping you to express data in the most intuitive way.In the Beta version you can Create and manage data models in Fusion and using a cli tool Ingest data into the data model using CDF transformations Query data using GraphQL with powerful search and filtering in Fusion or your own applications with even more powerful features

Hi there, I’m new to Cognite in general, but one task that my team is looking into is utilizing Cognite’s SVG files in our LCI and Plant Management applications and I have been looking into it somewhat.Not knowing all the details of the SVG rendition capabilities, I do see that our customer ‘Interactive P&IDs’ dataset in CDF has SVGs renditions without the comments that are in the PDF files (where applicable).Would Cognite be able to include the comments - now or in the future?The reason this is relevant for us is that a number of our doc profiles only have PDF files. Typically old documents on old installations.We’re missing native files (DGN, DWG), thus contractors are updating these drawings using comment features such as clouds, text, lines, etc in Adobe. That information seems to disappear in SVG renditions, but it’s quite critical information, especially in Plant Management systems. Thanks for listening to me!Best regards,Erik

Developing, tracking and meeting sustainability goals is becoming increasingly important for companies in the public sphere. Many of them are turning to data-driven solutions to help them monitor, report and reduce their environmental impact. Recent regulations in Europe recognises that Data-driven solutions for GHG emissions reductions are contributing to climate change mitigation (see official site from the European Commission - link) At Cognite, we have encountered many innovative solutions that promote sustainability. Examples include:  Automating the recording and reporting of greenhouse gas emissions in industrial processes. Optimizing energy efficiency at the equipment and system level to minimize scope 2 emissions. Using robotics to detect dangerous leaks. Leveraging operational data to derive the environmental footprint of discrete products. Protecting biodiversity with the automatic detection of birds close to onshore windmills  After analysing dozens of those solutio

I have found a problem reading timeseries from Cognite with Power BI. The first bit of Power Query has a filter to limit the timeseries retrieved to just the ones I want. The second one is the same, but does not have the filter. What I have found is that adding the filter causes duplicate rows in the table. Once the timeseries rows are duplicated, the timeseries aggregation values are also duplicated. I know that the first query is not the most performant option, but it should still have the correct answer.let Source = Cognite.Contents( #"cogniteContentsParameters", #"cdfEnvironment"), Timeseries_table = Source{[Name="Timeseries",Signature="table"]}[Data], #"Filtered Rows" = Table.SelectRows(Timeseries_table, each ([IsString] = false)), #"Removed Columns" = Table.RemoveColumns(#"Filtered Rows",{"MetaData", "SecurityCategories", "IsStep", "DataSetId", "CreatedTime", "LastUpdatedTime", "Asset", "Latest", "DataPoints", "StringPoints", "Aggregate"}), #"Filtered Rows1" = Tabl

Hi! After switching from experimental to cognite-sdk 3.9.0 I struggle with deployment. 1: Using handleIf i try to deploy using the handle, then the requirements are not handled. The function deploys, but failing at the imports. How should requiremenents be used when deploying like this? Imports are inside handle():def handle(client, data, secrets): from googleapiclient.discovery import build from googleapiclient.errors import HttpErrorCreating function like this:cognite_client.functions.create( name=function_name, external_id=function_ext_id, function_handle=handle)2 - Using a folder.Then I get this error: [WinError 267] The directory name is invalid: 'C:\\Users\\hknap\\AppData\\Local\\Temp\\tmptzkywbdl\\function.zip'If I debug, the folder “tmptzkywbdl” is created, so there should be no permission issues.  3- Manually uploading the zip fileThis works (including the requirements.txt), thanks for adding this possibility. It is just a little extra hassle ;)  would prefer if

Hi!Almost three years ago we made a feature request regarding search. Specifically we want to:Have support for paging. Do grouping on metadata-fields. Define which fields yields relevance to the search-results. Inclusive search-terms using AND instead of OR.I do understand that satisfying the search-requirements of all your users is a daunting task, but perhaps it would be easier to let us do the job instead? I believe it would be very beneficial if we were able to access ElasticSearch directly in some manner.And please keep in mind that the data itself defines relevancy. No two customers are alike and therefore a good generic solution likely doesn’t exist. @Knut Vidvei @Andreea Pastinaru 

Schedule a personalized demo to learn how Cognite Data Fusion™ generates fast, scalable value from your data, enabling better decision making about maintenance, production and safety.In the demo, we will:Understand your priorities, initiatives and challenges that you are looking to solve Introduce Cognite and share examples of use cases we deliver for our customers today Identify areas where Cognite Data Fusion can help your organizationRequest demo with one of our product experts 

If you are curious on how to get the most out of your Cognite Data Fusion subscription, you have come to the right place. This is part of a series of posts where we share some of our experience from working with customers in their journey towards an Industrial DataOps organization. We want to share lessons learned, mistakes made, good practices observed, and observations of pitfalls and risks. This is not the absolute truth, but hopefully a way to spark good discussions around an inherently complex topic!To quickly introduce ourselves, we are @Arjo Oosten, Digital Transformation Leader, winter sport addict and passionate about driving hands-on digital growth strategies and value based decision making, and @Karolina Luna, Solution Architect, cat lover, and passionate about the lifecycle perspective of everything (like solutions and data products). To learn more about Cognite Data Fusion, we recommend this post.Planning your solutions and Industrial DataOps with Cognite Data FusionTo be

Hello Charts Early Adopter Community,I know it’s been a bit quiet in this group lately, but rest assured, the team and I have been hard at work. We have just released several new features and fixes to production. You can scroll down to read about each of them in detail.Please do leave comments below with questions and feedback!What’s coming next?We’re in the middle of our development cycle in preparation for the next major release of Cognite Data Fusion in December 2022. There are the two major features our team is working on that you can look forward to having available:Moving Charts (charts.cogniteapp.com) into Cognite Data Fusion (fusion.cognite.com) We’ve received plenty of feedback that our product experience can and need to be more tightly integrated and we’ve heard you loud and clear. As you all know, Charts is currently available on charts.cogniteapp.com, which is separate from Cognite Data Fusion (fusion.cognite.com). Despite starting out on its standalone URL, Charts has alw

In many cases in diagrams including P&ID, there are shortcuts to represent tags, for instance:F2014A/B/C for F2014A , F2014B and F2014C.The contextualization engine in some cases will pick one of the 3 or none at all.Is there a way for the engine to detect trailing slashes and match to multiple tags?See example of patterns below:Example of patterns 

I am listing time series for a given asset, and get a lot of results. I need to filter based on the time series external ID, but the only option is by using the “external_id_prefix” argument to the list function. So I build up the prefix left to right. Somewhere in the external id is a parameter that I do not care about, and after comes a new parameter that I want a particular value of. Concrete example of external IDs:IAA_Miros_Weather_Data_WIA_008IAA_Miros_Weather_Data_WIB_008IAA_Miros_Weather_Data_WIC_008IAA_Miros_Weather_Data_WID_008IAA_Miros_Weather_Data_WIE_008I am interested in only getting time series with external IDs that contain “_WI” and that end with “008”. Is there a way to list time series with a wildcard? Something like this:client.time_series.list( asset_ids=[my_asset_id], limit=None, external_id_prefix="IAA_Miros_Weather_Data_WI*_008", partitions=4)I could of course obtain the relevant time series by filtering after the fact. Something like thisresult = cl

Are you a Power BI user who wants to gain hands-on experience with Cognite Data Fusion? Are you curious to learn how to handle a large set of CDF data in Power BI?The Cognite Power BI connector lets you use a CDF project as a data source in Power BI Desktop to query, transform and visualize data, share insights across your organization, or embed dashboards in your app or website. When working with the Cognite Power BI connector, it is important to know how to refresh the data and apply best practices to deliver accurate results. The newest course from Cognite Academy introduces you to implementing incremental refresh - one of the best practices to get the most out of the Cognite Power BI connector. Our instructors first explain when to use incremental refresh, then show you how to set it up in four steps. In this course, the training data comes from the Open Industrial Data (OID) project, a live stream of industrial data from the Valhall oil platform.After completing this course, you’l

Hi Team I am trying to deploy the code into new env facing issue because of black --check.