Product User Community

Hi, I see this error in extractor log, I hope even if this happens , it will be retried and there will not be the data loss. Please correct if I am wrong or if there are ways to change any configuration in extractor to avoid this.  2023-03-07 05:42:04.233 UTC [WARNING ] QueryExecutor_3 - Too many concurrent requests in pod | code: 503

In this post we’ll share with you our thoughts on key areas to focus on to empower you as Domain Experts - frontline workers - to efficiently utilize data and analytics in day-to-day operations. Let us know in the comments what you think!  The direction for Cognite Data FusionCognite Data Fusion (CDF) is an Industrial Data platform, and part of our mission is to ensure that we provide a “batteries included” experience to quickly realize value across our target industries Manufacturing, Energy and Power and Renewables.We want CDF to be the go-to tool for not only Data Scientist and deeply technical roles, but also the Domain Experts that know first hand what the operational challenges are and what the optimization potential is. Over the last couple of months, and in our upcoming releases, you will see a focus on enabling you as a Domain Expert to solve industrial data problems through low-code user interfaces - without assistance from “a coder”.  What sort of industrial data problems ar

It seems I can create an edge with any type. In the code example below, the externalId of the “type” is set to “xxx” when calling POST Add edges, which should not exist. However, the edge was added successfully. Furthermore, I could not see any edges were created in the SimulationModel table, even the “externalId” was set to “SimulatonModelSimulator”.  { "instanceType": "edge", "space": "XiaofengTest", "externalId": "externalId:edge1", "type": { "space": "XiaofengTest", "externalId": "xxx" }, "startNode": { "space": "XiaofengTest", "externalId": "externalId:simulator1" }, "endNode": { "space": "XiaofengTest", "externalId": "externalId:simmodel1" } } 

I’m trying to transform some time-series data (manufacturing pump pressures) into Asset Hierarchy/Numeric Data.The SQL is not writing to the timestamp column and not seeing errors in Preview.I’ve tried MANY different SQL commands to convert the DateTimeStamp string column to timestamp column/datatype and none have worked. CSV files are attached if anyone wants to try it. Added query in attachments. Help is greatly appreciated! Transform setup: Here is some of my data being edited in Excel and saved as CSV then imported into CDF Raw. TagID Tagname Value DateTimeStamp FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 NULL 3/2/2023 3:41 FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 28.54443 3/2/2023 5:51 FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 35.68053 3/2/2023 5:51 FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 42.81664 3/2/2023 5:51 FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 49.95274 3/2/2023 5:51  RAW     

Why are algorithms and data-driven models only available for the few domain experts who also are fluent in advanced software coding? We certainly don’t believe this should be the case.We are making them available to the rest of the world — particularly to non-coding domain experts.For many years  industrial data scientists have been building smart algorithms to solve  complex industrial problems. They are now available in a no-code drag-and-drop intuitive interface. Liberate your data and empower domain experts the tools to drive impact every day.Cognite Charts includes several data science toolboxes that provide subject matter experts (SMEs) out-of-the-box algorithms to process and manipulate data, conduct root cause analysis (RCA) and develop solutions without having to code.The toolboxes cover basic operations, statistical methods, data transformation, and advanced models. They work out-of-the-box with Cognite Charts, and we will continuously add new algorithms, features, and functi

It seems the link to download the demo data is not working in https://docs.cognite.com/cdf/data_modeling/upload_raw_dm. Once clicked the “the demo data” link, got an error message “This site can’t be reached”. Seems the dropbox link is broken. https://www.dropbox.com/s/o5o9eaewb9j57kt/FDM_Movie_data.zip?dl=1 Could you help have a check?

User sessions are managed via your IDP. Access token lifetime can vary from 60 to 90 minutes.So once the session expires, the user would normally have to sign-in again. How to overcome this?With the OIDC workflow, it is possible to retrieve a new access token without prompting the user to provide credentials again. This is done by finding a valid access token from cache or by finding a valid refresh token from cache and then automatically use it to redeem a new access token. The diagram below shows the normal OIDC workflow: Below you can find a sample code snippet which uses the acquire_token_silent method available through the class: msal.PublicClientApplication:def authenticate_azure(app): accounts = app.get_accounts() if accounts: print("Taking the token silently") creds = app.acquire_token_silent(SCOPES, account=accounts[0]) else: print("Taking token interactively") creds = app.acquire_token_interactive(scopes=SCOPES, port=PORT) return credsY

I have a transformation that is to write to a new dataset as seen here:SELECTconcat('CWS:',loc) as externalId,IF(parent_loc='' OR parent_loc IS NULL, '', concat('CWS:',parent_loc)) AS parentExternalId,CAST(lastUpdatedTime AS STRING) AS name,to_metadata(*) AS metadata,description AS description,7089382776719091 AS dataSetId FROM `CWS-Assets-DB`.`CWS-Assets-Tbl` The Preview seems to work fine.When RUN: I get following error:Request with id 96f4a082-9dde-9b41-8065-0b3cf0923197 to https://az-eastus-1.cognitedata.com/api/v1/projects/ra-istc-sandbox/assets/byids failed with status 403: Resource not found. This may also be due to insufficient access rights..The permissions on the dataset for my user group is raw:readraw:writeraw:listdatamodels:readdatamodels:writedatamodelinstances:readdatamodelinstances:writedatasets:readdatasets:writedatasets:ownertimeseries:readtimeseries:writefiles:readfiles:writeevents:readevents:writesequences:readsequences:write3d:read3d:create3d:update3d:deletetransfo

If  publishing a datamodel with TimeSeries attributes defined(like below)in the UI, it will fail with an error message:type Property {  modelName: String  value: TimeSeries} 

I am able to login with clientID and client Secret to our test project using cdf cli. Also the “cdf dm create” is working. However, when publishing a graphql using “cdf dm publish” there was always a validation error even using the demo GraphQL data model. Below is the error message:$ cdf dm publish --file=./test.graphql --external-id="testgraphql1" --space="testgraphql1" --version 1 --verbose2023-02-28T13:24:31.576Z cdf:middleware:auth Fetching auth token from CDF...2023-02-28T13:24:31.577Z cdf Getting token via Client Secret2023-02-28T13:24:31.578Z cdf:auth:client-secret Starting to aquire client credential token2023-02-28T13:24:32.032Z cdf:middleware:auth Finished fetching auth token from CDF2023-02-28T13:24:32.038Z cdf:data-models:publish dataModelVersionsHandler initialized2023-02-28T13:24:32.038Z cdf:data-models:publish Reading graphql file ./test.graphql2023-02-28T13:24:32.039Z cdf:data-models:publish Schema contents 'type Actor {\r\n' + '  name: String!\r\n' + '  age: Int\r\n'

A common setup in CDF (Cognite Data Fusion) is to first build an asset hierarchy, and then to attach events, timeseries etc. to those assets. One way to do it is to use transformations. In this short article, we’ll take events as an example. It works the same way for timeseries and sequences. Let’s assume we already have an asset hierarchy and we want to attach events to it. We’ll suppose that we have a RAW table, from which we want to create events, that looks like this:   In transformations, to attach an asset to a created event, you need to specify the corresponding asset ID (assetId in the target schema). Asset IDs being automatically generated, we usually prefer using external IDs because we know what they are made of: an ID from an ERP, a value following a naming convention, etc.  Since the asset ID is the expected value for the assetId field in the target schema, we need to retrieve for each asset its ID, based on its external ID. As mentioned in the documentation, we can read f

The coordinates of a document in CDF are normalized. In a normalized document, it has (0,0) coordinates in the upper left corner and (1,1) coordinates in the right bottom corner.Convert xml coordinates to normalized coordinatesTo convert xml coordinates to normalized coordinates you need to divide X coordinates with the width of the page and the Y coordinates with the height of the page. If the xml Y coordinate is 0 at the bottom of the page and the <height> is at the top of the page, the Y coordinate should be flipped as below.Normalized Y = 1- Normalized YOnce the coordinates are flipped;NewMinY = 1 - OldMaxYNewMaxY = 1 - OldMinY Example scenarioIf a user would like to add manual annotations to a P&ID file in CDF the user should convert the xml coordinates of the file to normalized coordinates. 

In Charts, we have been using the “Shift Time Series” function (for creating a new Calculation time series) to assess data more effectively, but have now noticed a limitation where the calculation is limited to the data from the source time series already displayed.  If you look at the first image below (just a 10-min shift), you can see in the green calculation that there is a gap on the left where the calculation doesn’t exist due to the source data being “off the page”.  If we go to a more extreme scenario (2nd image below), now of the data shows since we are attempting a 2+ day shift in a time series.  Is there a solution where this limitation can be removed?  For our scenario specifically, we are looking closely at (and comparing) three separate incidents that lasted about 1 minute each, but were spread out over the course of a week, so we are super zoomed in to look at the behavior signatures.  Thanks for the input!Kind regards,Richard10-minute shift​​​​​ Multiple-day shift 

I have asset hierarchy table and time series table imported, but I want to link asset to timeseries or vice-versa. asset hierarchy table looks like:level 1, level 2.e1, entity 1;level 1, level 2.e1, entity 2;level 1, level 2.e2, entity 3;time series table looks like:t1, entity 1, x1, y1t2, entity 1, x2, y2when I click asset entity 1 in Explore view, it shows time series data count but the name and description of the time series are empty; when click time series, the count of linked Assets is 0. I do not know how to fix the issue.  

Hello from the Solutions Portal team!We have a new release coming, which is by far our largest release yet! We hope this release will make it easier toBrowse through your data inside your CDF Project Enable new solutions built on top of CDF Enable new no code solution possibilities  We have a new home screen with new functionality!Any user can now ‘Browse solutions’. Through here, you can view all solutions that Cognite has to offer. If they’re not enabled, a demo can be requested. If you are an admin, SOME applications can be installed through a wizard for free (today, blueprint) and some are automatically installed (Charts).An explorer designed for SMEs. We’ve taken the old ADI explorer, made some improvements based on feedback, and now have an early version out for early adopters.An early version of global search is included also - easily search your way through any CDF resource (with more resources to come!) New CDF application: Blueprint!Blueprint is a no code solution that allows

I might need to create 100,000s - 1,000,000 of individual timeseries. I would like to understand if the number of timeseries impacts the API performance. e.g. retrieving data points for 1 of the timeseries. Would the API response be similar if I had 100 or 1 million timeseries?Also is there a maximum number of timeseries that can be created per CDF project?

Hi.Would it be possible to change the name format of the CSV file downloaded from cognite charts? The colon in “New chart 2023-02-22 15:43:59 (timeseries).csv” makes it difficult to unzip the file on Windows.

While using cognite-sdk-dotnet, it needs API key whereas I have client-id and secret with me. How can I get the API key for the same? Or there is some alternative way to work with client-id and secret.

I am trying new FDM APIs. In POST endpoints I am facing issue “Missing body for POST request”Am I missing some parameters? I am using POST body as per documentation 

Is there documentation for the import calculation json in Charts? 

Digitalization PoCs are commonplace. Real return on investment (ROI) isn’t.So how do investments in digital transformation efforts translate into real value for your business (or company)?We, at Cognite, commissioned a study from Forrester Consulting to examine the potential ROI and business benefits asset-heavy industrial organizations can expect from deploying Cognite Data Fusion. Forrester interviewed six representative customers across our customer base in Oil and Gas, Manufacturing, and Power & Cleantech with experience using Cognite Data Fusion, and found $21.6 million in added net present value at a 400% ROI. Key results of the ROI study include:* $9 million gained through the optimization of heavy machinery and industrial processes* $5.1 million saved through optimized energy use and reduced operational costs* $4.8 million added value through reduced shutdown time* $4.3 million cost reduction by optimizing planned maintenance  Do you want to know what stands behind these nu

I have set up both the environment variables SOURCE_CLIENT_SECRET (Note that the Client secret is generated from the OID widget as per the documentation)and DEST_CLIENT_SECRET and running the replicator with the config file using option 5.I am getting error as per below logAMAC02Z3123LVCJ:oid-replicator j.subhash.parandekar$ poetry run python3 ./oid_replicator/replicate.py 2023-02-17 18:46:48,239 root INFO - Config file - Repeat line 5: 2023-02-17 18:46:48,239 root INFO - Config file - Repeat line 14: 2023-02-17 18:46:48,239 root INFO - Config file - Repeat line 23: Starting replication of resourcesReplicating assets...Traceback (most recent call last):  File "/Users/j.subhash.parandekar/Library/Caches/pypoetry/virtualenvs/oid-replicator-cwaK-6ym-py3.11/lib/python3.11/site-packages/cognite/client/credentials.py", line 364, in _refresh_access_token    token_result = self.__oauth.fetch_token(                   ^^^^^^^^^^^^^^^^^^^^^^^^^  File "/Users/j.subhash.parandekar/Library/Caches/py

Using transformations to create/update data can be cumbersome especially for complex FDM models. I understand there are plans to generate APIs from models, but are there any alternatives in the short term, such as implementing GraphQL mutations? Are the APIs used by transformations internal to CDF?

I am running the notebook for my own Cognite project as per the example https://github.com/cognitedata/dshub-tutorials/blob/master/advanced/Comparing%20Entity%20Matching%20models%20with%20SDK%20demo.ipynbI am getting below error ModelFailedException: EntityMatchingModel 3531921825900910 failed with error 'JobFailedException: AttributeError('TfidfVectorizer' object has no attribute 'get_feature_names')' at line  results = model.predict(sources=time_series_test, targets=assets).result

Hi All, I am getting error while importing CogniteClient from cognite.experimental.Please refer the below screenshot of the error.