COGNITE HUB

Welcome to Our Community!

Join the conversation to shape a safer, more efficient, and sustainable industrial future.

393 Topics

L
Leandro SantosNew
asked in Developer and User Community

Numeric Data Points transformation not accept NULL values

I’m trying to Create a new Numeric DataPoint transformation and when I created a new transformation the UI shows a question on how it should handle NULL values (keep existing value or deleting).I’ve setup my query with a CAST function to convert values to double. I understand that this will return NULL values when CAST is not possible, so I would expect the transformation to follow the settings defined for the transformation, but instead the transformation fails with the following message:Column 'value' was expected to have type Double, but NULL was found Can you please clarify what is the expectation regarding the Numeric DataPoint Transformation. Can it have NULL values as input or not and what is the settings for NULL values about?Thanks

0
L
40 minutes ago
S
Committed
ShylashriSelvamaniCommitted
asked in Developer and User Community

cognite data fusion plugin connect to Grafana using Azure managed grafana -plugin issue

Hi, I have done the Grafana setup from azure managed grafana from azure portal. But don’t have permission to install cognite data fusion in grafana from azure. Please let me know how to install plugin and what permission needed.   

1
J
Practitoner
1 hour ago
joseLB
Active
joseLBActive
asked in Developer and User Community

Notebook 1_Authentication: Cannot authenticate to Azure

Hi: I’m doing DATA ENGINEER BASICS - INTEGRATE course.Trying to execute Notebook 1, I cannot pass through the cell getting the authentication with Azure, namely:creds = authenticate_azure()Any help?P.S. I’m using Google Colab  

3
joseLB
Active
3 days ago
S
Committed
ShylashriSelvamaniCommitted
asked in Developer and User Community

While python SDK: Cognite API error : Resource not found. This may also be due to insufficient access rights. | code: 403 | X-Request-ID: 941c405e-bd77-9105-9485-4bfe9575c998

We are trying to connect CDF Project instance : accenture-demo-dev through python SDK. We are getting the below error :Cognite API error : Resource not found. This may also be due to insufficient access rights. | code: 403 | X-Request-ID: 941c405e-bd77-9105-9485-4bfe9575c998We have provided all capabilities access to this project. cdf project link : https://accenture.fusion.cognite.com/accenture-demo-dev?cluster=api.cognitedata.com    

11
H
Practitoner
3 days ago
S
Active
samirhajActive
asked in Developer and User Community

detNet RestAPI Cognite SDK example

Hello,I am quite new on usinf RestAPI. My company has the license to use CDF. I also inderstood that Cognite has developed dotNet RestApi SDK.I searched a bit but I could not find an example of how we can use this SDK for data retrieval especially for time-series data. How could I get a snippet of code showing an example how to use this?Thanks. 

3
Everton Colling
Practitoner
4 days ago
N
Active
N G NidhiActive
asked in Developer and User Community

Azure pipeline failure

Hi Team, I am facing some issues while using the above pipeline.yaml  file setup which is suggeted by cdf team before, i am getting validation error, even after passing all mandatory fields.Could you please help me doing the devops setup so that i can understand the process.Please set up a call to work on this issue as it is impacting deliverables. Regards,Nidhi N G

1
C
Practitoner
5 days ago
N
Committed
nabanitakarCommitted
asked in Developer and User Community

Can I use Asset Hierarchy in Grafana

My Question is : Is it possible to use  Asset Hierarchy in Grafana?I found many things related to timeseries  but not related  Asset Hierarchy to Grafana. Thanks. 

6
N
Committed
3 days ago
jasar
Active
jasarActive
asked in Developer and User Community

Error when run a transformation from the course.

i need support for the course Python SDK Transformations.Im running the comands from notebook that was given in git and a stoped on:result = client.transformations.run(asset_transformation.id, wait=False)The error resulted is :CogniteAPIError: Invalid source/destination credentials: Could not authenticate with the OIDC credentials. Please check your credentials. | code: 403 | X-Request-ID: 78e0e54a-a3ca-9bc4-b2f5-e303ed6a7207iv checked the authentication process and i can perform all task like search list add datasets assets and everything but i cant run the transformation.Any one knows how to solve it ?i have the same problen in the UI  

9
jasar
Active
4 days ago
A
Active
Alex NarayananActive
asked in Open Industrial Data

Realtime use cases vs near realtime use cases

I’ve been introduced to CDF in the last 2 weeks, very interesting and learning through. Just wondering if CDF has been used for real time monitoring use cases, vs. predictive/BI type use cases. For example, how soon can the alerts be triggered from the time a new abnormal measurement is detected. I imagine that it depends on a lot of factors  such as network latency, source extractors, transform functions & schedules, Is there a threshold (say 1hr response time) beyond which CDF can be used.

2
A
Active
4 days ago
Anders Brakestad
Seasoned
Anders BrakestadSeasoned
asked in Developer and User Community

How to ensure identical DateTimeIndex in TS data from CDF and Pi (via Seeq API)

I am trying to make a comparison between time series data from CDF and from Pi (accessed with the Seeq Python API). Some questions arise:​​​​​How do I ensure that the DateTimeIndex-s that are returned are identical? Why does CDF return DateTimeIndex with dates before my start value?Examples:Pulling a raw time series from Pi via Seeq looks like this:start = datetime(2021, 1, 1)end = datetime(2021, 1, 2)spy.pull(items, start=start, end=end, grid=None, quiet=True)And the CDF analog looks like this:start = datetime(2021, 1, 1)end = datetime(2021, 1, 2)cognite.datapoints.retrieve_dataframe( external_id=external_id, start=start, end=end, granularity=None)The resulting DateTimeIndex-es are not identical:From Pi via Seeq:DatetimeIndex(['2021-01-01 00:00:00+01:00', '2021-01-01 00:00:05+01:00', '2021-01-01 00:00:10+01:00', '2021-01-01 00:00:15+01:00', '2021-01-01 00:00:20+01:00', '2021-01-01 00:00:25+01:00', '2021-01-01 00:00:30+01:00', '2021

7
Anders Brakestad
Seasoned
6 days ago
N
Active
N G NidhiActive
asked in Developer and User Community

azure devops setup

Hi Team, I am facing some issues while using the above pipeline.yaml  file setup to run the pipeline, Could you please help me doing the devops setup so that i can understand the process.I will set up a call , please let me know your available timings. Regards,Nidhi N G

1
D
Practitoner
4 days ago
eberribeiro
Active
eberribeiroActive
asked in Open Industrial Data

OSDU and OID

Hi, I would like to understand the relationship of OSDU x OID? Are they competitors, the differ on, pros x cons.. Thanks in advance..

3
eberribeiro
Active
7 days ago
K
Seasoned
Karthik KaliyaperumalSeasoned
asked in Open Industrial Data

Learn 360 Images

Learn 360 images in Cognite Reveal.

5
K
Seasoned
4 days ago
Chris Selph
Seasoned
Chris SelphSeasoned
asked in Developer and User Community

Transforms Creating Nulls for timestamp

Hi, I’m running a transform from Raw to insert/update Numeric values./* MAPPING_MODE_ENABLED: false *//* {"version":1,"sourceType":"raw","mappings":[{"from":"","to":"id"},{"from":"","to":"externalId"},{"from":"","to":"timestamp"},{"from":"","to":"value"}]} */SELECT dp.Tagname AS externalId,to_timestamp(DateTimeStamp, 'yyyy-M-d h:mm:ss') AS timestamp, cast(value AS double) FROM `ISTC_WW_InSQL`.`wwHistory` AS dp , --selecting from _cdf.timeseries means we select from the timeseries we ingested to CDF earlier. We do this to make sure all the time series we try to add data points to actually exist_cdf.timeseries AS ts WHERE dp.`Tagname` = ts.externalId--And to_timestamp(DateTimeStamp, 'yyyy-M-d h:mm:ss') Is Not Null  When previewing the code above, you can see some null values. Also, if you “run” the code it complains about nulls. (Not pictured.)   I think I’ve confirmed that the DateTimeStamp column in `ISTC_WW_InSQL`.`wwHistory` contains no null values.When, we UnComment the last line

2
Chris Selph
Seasoned
3 days ago
Sofie Svartdal Berge
Practitoner
Sofie Svartdal BergePractitoner
posted in Developer and User Community

Cognite Data Fusion for you as a Domain Expert

In this post we’ll share with you our thoughts on key areas to focus on to empower you as Domain Experts - frontline workers - to efficiently utilize data and analytics in day-to-day operations. Let us know in the comments what you think!  The direction for Cognite Data FusionCognite Data Fusion (CDF) is an Industrial Data platform, and part of our mission is to ensure that we provide a “batteries included” experience to quickly realize value across our target industries Manufacturing, Energy and Power and Renewables.We want CDF to be the go-to tool for not only Data Scientist and deeply technical roles, but also the Domain Experts that know first hand what the operational challenges are and what the optimization potential is. Over the last couple of months, and in our upcoming releases, you will see a focus on enabling you as a Domain Expert to solve industrial data problems through low-code user interfaces - without assistance from “a coder”.  What sort of industrial data problems ar

0
Sofie Svartdal Berge
Practitoner
13 days ago
N
Committed
Niranjan Madhukar KarvekarCommitted
asked in Developer and User Community

DB Extractor - Too many concurrent requests in pod

Hi, I see this error in extractor log, I hope even if this happens , it will be retried and there will not be the data loss. Please correct if I am wrong or if there are ways to change any configuration in extractor to avoid this.  2023-03-07 05:42:04.233 UTC [WARNING ] QueryExecutor_3 - Too many concurrent requests in pod | code: 503

1
M
Practitoner
13 days ago
K
Active
kevinpengActive
asked in Developer and User Community

cognitedata / cognite-sdk-js

https://github.com/cognitedata/cognite-sdk-js/tree/master/samples/react/msal-browser-react, I am trying to run npm install and $ REACT_APP_CDF_PROJECT=... REACT_APP_AZURE_TENANT_ID=... REACT_APP_AZURE_APP_ID=... npm startgot the following error:node:internal/modules/cjs/loader:1024  throw err;  ^Error: Cannot find module '/Users/kevin.peng/code/cognite/cognite-sdk-js/samples/react/msal-browser-react/start'    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1021:15)    at Function.Module._load (node:internal/modules/cjs/loader:866:27)    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)    at node:internal/main/run_main_module:22:47 {  code: 'MODULE_NOT_FOUND',  requireStack: []}

3
B
Practitoner
11 days ago
N
Active
N G NidhiActive
asked in Developer and User Community

Cognite(CDF) Function for azure DevOps setup

Hi Team, Please help with the current setup or process  of azure devops  to deploy Cognite (CDF) Function(API) to CDF project.

2
N
Active
6 days ago
Chris Selph
Seasoned
Chris SelphSeasoned
asked in Developer and User Community

Transformation to Numeric Data

I’m trying to transform some time-series data (manufacturing pump pressures) into Asset Hierarchy/Numeric Data.The SQL is not writing to the timestamp column and not seeing errors in Preview.I’ve tried MANY different SQL commands to convert the DateTimeStamp string column to timestamp column/datatype and none have worked. CSV files are attached if anyone wants to try it. Added query in attachments. Help is greatly appreciated! Transform setup: Here is some of my data being edited in Excel and saved as CSV then imported into CDF Raw. TagID Tagname Value DateTimeStamp FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 NULL 3/2/2023 3:41 FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 28.54443 3/2/2023 5:51 FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 35.68053 3/2/2023 5:51 FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 42.81664 3/2/2023 5:51 FCE1-HYD-PMP-3 FCE1-HYD-PMP-3 49.95274 3/2/2023 5:51  RAW     

3
G
Seasoned Practitioner
14 days ago
J
Active
Jerry DabrowskiMowi
asked in Developer and User Community

Transformation SQL - check presence of a key in asset metadata

Hi all,what's the best way to write a condition in the transformation query, that checks for presence of certain key in asset metadata?Example asset metadata:"KEY": "VALUE"pseudocode:select * from _cdf.assets A where ‘KEY’ in A.metadata.keys I’ve been provided with following solutiion by Cognite support, with a note to ask here. Is there a better/other solution?select * from _cdf.assets where metadata['KEY'] is not null;

2
J
Active
13 days ago
K
Practitoner
Kanchana PremachandraPractitoner
posted in Developer and User Community

Acquiring an access token without user interaction

User sessions are managed via your IDP. Access token lifetime can vary from 60 to 90 minutes.So once the session expires, the user would normally have to sign-in again. How to overcome this?With the OIDC workflow, it is possible to retrieve a new access token without prompting the user to provide credentials again. This is done by finding a valid access token from cache or by finding a valid refresh token from cache and then automatically use it to redeem a new access token. The diagram below shows the normal OIDC workflow: Below you can find a sample code snippet which uses the acquire_token_silent method available through the class: msal.PublicClientApplication:def authenticate_azure(app): accounts = app.get_accounts() if accounts: print("Taking the token silently") creds = app.acquire_token_silent(SCOPES, account=accounts[0]) else: print("Taking token interactively") creds = app.acquire_token_interactive(scopes=SCOPES, port=PORT) return credsY

0
K
Practitoner
18 days ago
M
Committed
Mohit ShakalyCommitted
asked in Developer and User Community

cognite-sdk-dotnet in production

Is there any known limitation in cognite-sdk-dotnet to be consumed in enterprise solution?

3
M
Committed
7 hours ago
Chris Selph
Seasoned
Chris SelphSeasoned
asked in Developer and User Community

Transformation Run Error

I have a transformation that is to write to a new dataset as seen here:SELECTconcat('CWS:',loc) as externalId,IF(parent_loc='' OR parent_loc IS NULL, '', concat('CWS:',parent_loc)) AS parentExternalId,CAST(lastUpdatedTime AS STRING) AS name,to_metadata(*) AS metadata,description AS description,7089382776719091 AS dataSetId FROM `CWS-Assets-DB`.`CWS-Assets-Tbl` The Preview seems to work fine.When RUN: I get following error:Request with id 96f4a082-9dde-9b41-8065-0b3cf0923197 to https://az-eastus-1.cognitedata.com/api/v1/projects/ra-istc-sandbox/assets/byids failed with status 403: Resource not found. This may also be due to insufficient access rights..The permissions on the dataset for my user group is raw:readraw:writeraw:listdatamodels:readdatamodels:writedatamodelinstances:readdatamodelinstances:writedatasets:readdatasets:writedatasets:ownertimeseries:readtimeseries:writefiles:readfiles:writeevents:readevents:writesequences:readsequences:write3d:read3d:create3d:update3d:deletetransfo

3
G
Seasoned Practitioner
18 days ago
J
Committed
jaydeepCommitted
posted in Developer and User Community

Cognite replicator fails to replicate asset data with inconsistent behaviour

The cognite replicator fails to replicate asset data and the behaviour is inconsistent. It worked yesterday and not working today. Note that all resource types like time series, events and datapoints are replicated consistently. I am getting below error while replicating the asset data.Please see attached screenshot for the successful run of the asset replication yesterdayTraceback (most recent call last):  File "/Users/j.subhash.parandekar/oid-replicator/oid_replicator/cognite_replicate.py", line 84, in <module>    assets.replicate(SOURCE_CLIENT, DEST_CLIENT, config=cognite_config,  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/cognite/replicator/assets.py", line 324, in replicate    src_dst_ids_assets = create_hierarchy(                         ^^^^^^^^^^^^^^^^^  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/cognite/replicator/assets.py", line 220, in create_hierarchy    updated_assets = replication

2
G
Seasoned Practitioner
4 days ago
Pierre Pernot
Practitoner
Pierre PernotPractitoner
posted in Developer and User Community

How to create events for different assets with transformations

A common setup in CDF (Cognite Data Fusion) is to first build an asset hierarchy, and then to attach events, timeseries etc. to those assets. One way to do it is to use transformations. In this short article, we’ll take events as an example. It works the same way for timeseries and sequences. Let’s assume we already have an asset hierarchy and we want to attach events to it. We’ll suppose that we have a RAW table, from which we want to create events, that looks like this:   In transformations, to attach an asset to a created event, you need to specify the corresponding asset ID (assetId in the target schema). Asset IDs being automatically generated, we usually prefer using external IDs because we know what they are made of: an ID from an ERP, a value following a naming convention, etc.  Since the asset ID is the expected value for the assetId field in the target schema, we need to retrieve for each asset its ID, based on its external ID. As mentioned in the documentation, we can read f

1
G
Seasoned Practitioner
19 days ago

Generate client secret for OID

Please generate a client secret once every 180 days.




Badge winners

Show all badges
Terms & ConditionsCookie Settings