Explore user resources including how-to guides and video tutorials
hello Community,can anyone help me understand how to upload P&ID diagram files to CDF, Using Python SDK.
hello Community,Can someone please share the approach (along with code) for extracting files from sharepoint online (xls , with multiple worksheets) and extract the content and load them as RAW tables in CDF. Is there a direct feature available in Sharepoint file extractor that does this job? Should we use SDK to extend the file extraction and then read the content and insert into tables?. If we have a large set of files and each containing multiple sheets, it can be hard to process all of them dynamically. Please advise.
Hello,I am quite new on usinf RestAPI. My company has the license to use CDF. I also inderstood that Cognite has developed dotNet RestApi SDK.I searched a bit but I could not find an example of how we can use this SDK for data retrieval especially for time-series data. How could I get a snippet of code showing an example how to use this?Thanks.
I created a test data model in a test space. I finished testing and deleted the data model, but can not find a way to delete the space (which no longer contains any data models)?
hello community, can anyone tell me is there any way to upload .csv files to CDF using Python SDK.
How can I filter assets by their geolocation using the python SDK?from cognite.client.data_classes import GeoLocationFilter, GeometryFiltermygeofilter = GeoLocationFilter( relation="within", shape=GeometryFilter( type="Polygon", coordinates=[[[7, 61], [13, 61], [10, 55], [7, 61]]] ))client.assets.list(geo_location=mygeofilter)This approach works for files, but for assets I get:CogniteAPIError: Unexpected value 'within' | code: 400 | X-Request-ID: fd1c5a73-a3d3-9ec1-bd97-039b4b1e0051
Hi Team,I am facing one challenge in creating Asset hierarchy in Power BI. Format of data is not as per the required format for hierarchy creation i.e. i do not get parent-child relation in different columns. However i see all nodes appear in single column. I do have ParentID but unaware how this will help me to form hierarchy. Any inputs will be appreciated. Thanks!Regards,Arati
(RAW Table : As above which has total 179 columns #41 contains numeric data 138 contains text data)I have ingested data in raw table….now I want to get only data of 41 columns which has number data type ?how this can be done in the transformation?similar query I want to run for remained 138 columns which has “Text” data type?
How to unpivot any existing raw table?I want to transpose columns to rowshere I am aware about stack can be used, but i dont want to give hard coded columns names... how this can be done run time (Without passing any hardcoded value of column name in stack command?)How to get run time columns count & how this columns can be passed in stack? Table1:id p1 p2 p3 1 2.0 3.0 4.0 2 10.0 15.0 20.0 select id, stack(3, 'p1', p1, 'p2', p2, 'p3', p3) as (property,value)from `db1`.`Table1`
With the introduction of the Documentum Extractor version 2.0.0 onwards, it’s required to add the dfc.jar file into the lib folder. So to be able to run as a jar file user must add the dfc.jar file into the lib folder. The newly introduced Documentum Extractor is built against version 7.2.0180.0170 of DFC.Issue users might face when running the jar file with the below commandjava -jar path/to/documentum-extractor-<version>.jar path/to/config.yamlIf a user tried to run the Documentum Extractor with the dfc.jar file with the above command as mentioned in this guide, the user will receive an error message similar to the:Main - Documentum Extractor failed:java.lang.NoClassDefFoundError: com/documentum/fc/common/DfExceptionThis is due to the fact that the Documentum Extractor jar is not identifying the main classpath.Workaround to run the Documentum Extractor with the dfc.jar file.As a workaround users could specify the classpath to include the extractor and run it as shown below. Wi
R is a programming language and environment for statistical computing and graphics. It was developed in the 1990s by statisticians Ross Ihaka and Robert Gentleman and has since become one of the most widely used programming languages for data science. R is an open-source project, which means that anyone can contribute to its development and use it for free.R is particularly popular among data scientists because of its powerful built-in tools for data manipulation, analysis, and visualization. R has a wide range of packages that can be used to perform various tasks in data science, such as data cleaning, statistical modelling, machine learning, and data visualization. R also has a highly active and supportive community, which means that there are many resources available online for learning and troubleshooting.Fetching live data from Cognite Data Fusion in R can provide data scientists and analysts with a powerful tool for analysing and visualizing industrial data. It can help them iden
If you have a sandbox that you would like to populate with data to get started with Cognite Data Fusion, you can use Open Industrial Data. We have built a Cognite replicator for that purpose. Disclaimer: This component is Community content and not officially supported by Cognite. Bugs and changes will be fixed on a best effort basis. Feel free to open issues and pull requests, we will review them as soon as we can. How to use the Cognite replicator? You have detailed instructions on Github. You have information on how to run it as python package, docker container or a standalone script and authentication via user impersonation and client credentials. How do I authenticate to my source and destination project? What permissions do I need? You have 2 ways of authenticating to your source and your destination:via user impersonation. the script will be run on behalf of your user, you should therefore have the required permissions (read/write on the resources you want to write to/from + gro
Introduction to Asynchronous Programming in Python SDKAsynchronous programming in Python is a powerful way to handle multiple tasks at once, without blocking the execution of the main program.First, let's start by understanding what asynchronous programming is and how it works. In traditional synchronous programming, a program will execute one task at a time, waiting for each task to complete before moving on to the next. This can be very slow and inefficient when dealing with large amounts of data or slow-performing tasks.On the other hand, asynchronous programming allows a program to execute multiple tasks at the same time, without waiting for each task to complete before moving on to the next. This is achieved by using a technique called "non-blocking I/O," which allows the program to continue executing other tasks while waiting for input or output. The below image gives a better understanding of the difference between synchronous and asynchronous functions. In this Hub post, we wil
Has anyone configured the extractor to use Visual Studio Code? I need a step by step guide.
IN SHORT This article aims to showcase how you can get value from your sensor data by continuously monitoring the health condition of a rotating equipment. We will present details of a health monitoring method that is being used by multiple operators across the globe to monitor their high value equipment. Failure of a rotating equipment, such as a turbine, pump or a compressor can cost an operator millions of dollars in revenue and repair. These assets generally contain a large number of sensors that capture the vibration, pressure, temperature and other useful metrics in a live data stream. When the data coming from the sensors go through appropriate analytics, it can correctly categorize the “health risk level” of an equipment. Operators can monitor a live dashboard that continuously updates the asset risks or subscribe to alerts that send a notification whenever there is concern in the operations. METHODThe asset health risk is based on the percentage of anomalies in the incoming d
Automated tests are considered a good practice in software development. They can apply to every language or use case. Here we will focus on Python and testing code that involves the Python SDK’s CogniteClient object. Below are listed some of the benefits of testing. First, they are a good way to test your code in a fast and repeated manner. With automated tests, you avoid testing manually your code: you can write each test case programmatically and run them on request. When they fail, running them in a debugger really helps understand the code, each object’s structure, and solve the problem quite fast. Then, it helps make the code maintainable. Indeed, when the code needs to be changed, running your tests ensures that the tested code will still work as expected. When a bug is discovered, it is good to create a test case for that bug to avoid having it again when updating the code. It also helps collaborate on the same repository. When several people work on the same code, it might hap
Leaving this as a how to guide for others that struggled like I have been.I am in the position where I want to use the publicdata with Python using a ClientSecret. I did not want to use interactive logons or the device flow. I struggled to understand exactly which bits of the SDK and which values to use where.Working Python code that will run in a Django management command is below but it should be trivial to change it for other apps.import asyncioimport atexitimport datetimeimport osimport timefrom channels.layers import get_channel_layerfrom cognite.client import ClientConfig, CogniteClientfrom cognite.client.credentials import OAuthClientCredentials, Tokenfrom django.core.management.base import BaseCommandfrom msal import PublicClientApplication, SerializableTokenCache# Contact Project Administrator to get theseTENANT_ID = "48d5043c-cf70-4c49-881c-c638f5796997" # use Cognite Client details from https://hub.cognite.com/open-industrial-data-211/openid-connect-on-open-industrial-data-9
This guide describes how to run the Cognite dB Extractor in a separate Docker container to fetch data from Postgres server to Cognite Data Fusion. Prerequisites Running instance of PostgreSQL server and valid credentials to SELECT rows from the table in question Docker host that can access the PostgreSQL server Running CDF project with an Azure AD service principal with capabilities to write either to CDF. 1 - Prepare Docker ImageCognite provides the database extractor as a Docker image published to Docker Hub, requiring just the addition of an ODBC driver. Since we are connecting to PostgreSQL we will install drivers provided by PostgreSQL for Debian 11, using this Dockerfile: Note! Go to https://hub.docker.com/r/cognite/db-extractor-base/tags to see the latest version of the Docker imageFROM cognite/db-extractor-base:2.5.0-beta5RUN apt-get install odbc-postgresqlBuild the image: docker build -t cognite-db-postgres . 2 - Setup extractor configThe extractor will read a local conf
REST APIs are one of the most common ways to expose data. At Cognite we often encounter use cases where we have to extract data from such APIs (for example extracting data from SAP) to Cognite Data Fusion (CDF). Since every REST API is totally custom, we do not have an off-the-shelf REST extractor. Nevertheless, we have a package called extractor-utils enabling you, among others, to create custom extractors for REST APIs: https://cognite-extractor-utils.readthedocs-hosted.com/en/latest/ In this article, we’ll go through an example using Meteorologisk Institutt’s Frost API, exposing meteorological data from Norway. It is a free API, the documentation is available at https://frost.met.no/api.html#/locations . You need to create a user in order to use the data: https://frost.met.no/howto.html . The goal will be to create an asset and a timeseries for each one of the locations. Then we will feed datapoints (for the meteorological reports) inside of those timeseries thanks to an extractor.
As Support team we can sometimes ask you to share a .HAR file when you report an issue in a web application.HAR is the short form for HTTP ARchive format, which tracks all the logging of web browser's interaction with a site.It will save time if you already attach this file when you reach out to us initially. If you are experiencing any of the below issues, a .HAR file will be useful: Performance issues: slow page loading, timeouts when performing certain task Page rendering issues: incorrect page format, missing information Issues in the UI If you want to know how to generate a file like that, keep on reading! Generating a .HAR file for Google Chrome Open Google Chrome and go to the page where the issue is occurring. Look for the vertical ellipsis button and select More Tools > Developer Tools. From the panel opened, select the Network tab. Look for a round record button ( 🔴 ) in the upper left corner of the tab, and make sure it is red. If it is grey, click it once
This guide describes how to run the Cognite dB Extractor in a separate Docker container to fetch data from Microsoft SQL server to Cognite Data Fusion. Prerequisites Running instance of MS SQL server and valid credentials to SELECT rows from the table in question Docker host that can access the MS SQL server Running CDF project with an Azure AD service principal with capabilities to write either to CDF. 1 - Prepare Docker ImageCognite provides the database extractor as a Docker image published to Docker Hub, requiring just the addition of an ODBC driver. Since we are connecting to MS SQL we will install drivers provided by Microsoft for Debian 11, using this Dockerfile: Note! Go to https://hub.docker.com/r/cognite/db-extractor-base/tags to see the latest version of the Docker imageFROM cognite/db-extractor-base:2.5.0-beta5RUN curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \ curl https://packages.microsoft.com/config/debian/11/prod.list > /
Hi all!I am trying to obtain weather data from Miros for a given asset ID. I am experienced in Python, but new to CDF, and don’t have access to a lot of practical examples of common tasks and patterns. So I thought I’d ask here to speed up my work :) What would be best practice for the given scenario (given in detail to try to avoid the XY problem)?I am going to work on some root cause analyses, and the starting point is likely to be an asset that failed multiple times for the same reason in the past. I want to fetch different types of data for this asset, such as maintenance history, relevant metadata, and weather data at its location during the relevant time window. This will eventually make its way into a simple graphical tool that I can use at the start of new root cause analyses.I have been playing around with retrieving an asset object based on its id or external_id. This seems to be fine. Then I notice that weather time series from Miros are connected to assets. But how would I
Check the
documentation
Ask the
Community
Take a look
at
Academy
Cognite
Status
Page
Contact
Cognite Support
