Developer and User Community

Join the discussions about delivering apps and data-driven decisions powered by Cognite Data Fusion. Click the + CREATE TOPIC button in the menu bar to start the conversation.

  • 692 Topics
  • 2,167 Replies

692 Topics

L
Seasoned
Lucas Carvalho de SousaSeasoned
asked in Developer and User Community

[FDM] How to filter custom array properties in a GraphQL query

I have some views with simple array properties (arrays of primitives) and others with arrays of relationships to views (edges).Location viewI can filter the simple arrays with the “containsAll” or “containsAny” methods.But there seems to not be a filter avaiable for the edges in the Graphql API. How can I filter by these properties via the GraphQL API?

4
L
Seasoned
8 months ago
A
Committed
Amit RaneCommitted
asked in Developer and User Community

Getting Invalid argument while running query in extractor

Getting OSError: [Errno 22] Invalid argument while I am running a query while other queries are running.And also this query is running in SQL server only giving error of invalid argument while ingesting timeseries data in extractor. 

1
C
Seasoned Practitioner
9 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

Timeseries creation - how to make it faster

Hi I have a final computed dataframe that looks like this. sample set alone listed here (15 rows) There are 570 records. I need to create timeseries for sumprod and derivedYields columns as per the value in Yieldcode. I have written the code and it is running and timeseries is getting created. But it is running slow. How to make it get done faster?Any best method? Please advise     sumprod Yieldcode Rank Group derivedYields MBBLD Vol LP-Class 1 532 0.009636 VBALFG1 1 VBAL 0.009636 3.524946 FG1 2 533 0 VBALLP1 2 VBAL 0 0 LP1 3 534 0.210269 VBALFN1 3 VBAL 0.218807 80.0382 FN1 4 535 0.037774 VBALLW1 4 VBAL 0 0 LW1 5 536 0.048263 VBALSK1 5 VBAL 0.094867 34.70158 SK1 6 537 0.017368 VBALHK1 6 VBAL 0 0 HK1 7 538 0.128149 VBALSR1 7 VBAL 0.142624 52.17094 SR1 8 539 0.014722 VBALDS1 8 VBAL 0 0 DS1 9 540 0.034342 VBALAG1 9 VBAL 0.0345

7
H
Practitioner
8 months ago
L
Active
Lanka Dharma SriharshaActive
asked in Developer and User Community

config import error

I’m facing the above issue while running the demo cognite custom extractor

1
C
Seasoned Practitioner
9 months ago
R
Active
rupaliActive
asked in Developer and User Community

Cognite Prosper Connector - Simulator Details

Hello Team,I am working on creating POC on PROSPER Connector I checked documentation of Cognite Prosper Connector. Connector is provided by Cognite, but where can I get simulator for POC purpose?Also in connector configuration, where to provide simulator details or simulator name?  

2
N
4 months ago
S
Active
SHAHNAZ P MOOSAActive
asked in Developer and User Community

How to configure Extraction pipeline for in-built CDF extractors supports

We have some requirements to use DB in built extractors and how can we setup Extraction pipeline configuration. 

3
S
Active
8 months ago
X
Seasoned
Xiaofeng WangSeasoned
asked in Developer and User Community

Filtering based on externalID

Hi,To apply a startWith filter on a property, we can do:prop_view = client.data_modeling.views.retrieve(('myspace', 'myview'))filters.Prefix(prop_view[0].as_id().as_property_ref("myPropertyName"),"xxx")This is working fine. However, as the externalId is not a property so the code below doesn’t work:filters.Prefix(prop_view[0].as_id().as_property_ref("externalId"),"xxx")and the error message is: “Invalid prefix filter with property reference [myspace, myview/94b12855a09cf4, externalId]. View myspace:myview/94b12855a09cf4 does not have property externalId.” Could you help find out how to apply a startWith filter on externalId in python sdk? Regards,Xiaofeng

8
X
Seasoned
8 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

read a csv file from Files in CDF as a dataframe

I am looking forward to a solution for reading a csv files that has 100 columns and 200 rows. It is stored in CDF as ‘Files’. How do I read the file and then use pandas to convert the file as a dataframe?diet_f= (client.files.list(name='diet_daily.csv'))[0]diet_daily_transformed = pd.read_csv(diet_f.name)diet_daily_transformedThis isnt working and throwing an error. Please advise the best means to perform this task. 

5
E
Practitioner
9 months ago
P
Seasoned
Peter QuinnSeasoned
asked in Developer and User Community

Python API calls work in Jupyter Notebook, but fail when running as a Python script

I have some code working great via a Notebook, but I want to move it to a server where I can run it as daemon. The authentication is failing when I run it on the server.I suspect the problem is with the Scopes - User impersonation might work in the browser where I’m already logged in to the right project. What scope should I use instead when I’m calling directly from Python?Also tried myscope = '[f"https://westeurope-1.cognitedata.com/.default"]'as suggested in the documentationoauthlib.oauth2.rfc6749.errors.AccessDeniedError: (access_denied) Unauthorizedfrom cognite.client import CogniteClient, ClientConfigfrom cognite.client.credentials import OAuthClientCredentialsfrom cognite.client.data_classes import TimeSerieshost = "https://westeurope-1.cognitedata.com"myproject = "rok-buandcollaborators-53"clientid = "realStuffWasHere"secret = "secretsecretsecretsecretsecretsecret"#myscope = "https://westeurope-1.cognitedata.com/.default"myscope='["IDENTITY", "user_impersonation"]'token="https

4
P
Seasoned
9 months ago
P
Seasoned
Peter QuinnSeasoned
asked in Developer and User Community

How to add externalId to a Timeseries

I created a set of timeseries via the API. The Python examples do not show how to include an externalId nor do they say one is really needed. Looks like they need one, otherwise they don’t work in Charts. That’s all fine. My problem now is to add the externalId to existing timeseries.I think the right way to do this is with a transform, but I can’t get it right. I can get it to preview correctly, but it fails when I run it.My transform is:Select name as name, concat(name,"123") as externalIdfrom `_cdf`.`timeseries` where dataSetId = 8883655420989208;The error is Request with id 2463250f-1635-9a05-aa9a-709366e80349 to https://westeurope-1.cognitedata.com/api/v1/projects/rok-buandcollaborators-53/timeseries/update failed with status 400: Time series not found. Missing externalIds: [Tank-401-01-TT123].Looks like it’s trying to query for the timeseries with that externaId when I’m trying to set the externalId. How do I get it to set the value?

4
C
Seasoned Practitioner
9 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

Timseries linked to an asset

How to fetch all the timeseries linked to an asset? I have a root asset that has 16 child assets. Each of those child has 18 timeseries objects. How do I write a code so that I can loop through this setup and read each of those timeseries objects and read the datapoint for a particular day and gather them. root = client.assets.list(name='liquid_asset')children=client.assets.list(parent_ids=[root[0].id])for i in children: print(i.time_series)Result is this: <bound method Asset.time_series of <cognite.client.data_classes.assets.Asset object at 0x3468500>>I could not iterate through the i.timeseries and read the datapoint from a very specific timeseries object named as 'PVOL'. Please help. 

3
H
Practitioner
9 months ago
X
Seasoned
Xiaofeng WangSeasoned
asked in Developer and User Community

Unstable call for the instance creating

Hi,We saw quite frequently an error message like below:"could not add instances: 500 - Internal server error. Please report this error to support@cognite.com and provide us with your request id: 4594f717-51c0-9708-86d3-5c5431f60cd1 and cluster name: westeurope-1"But when run it again it worked. The issue happens quite often(the number of instances in the request is close to 1000), could you have a check? Thanks. Regards,Xiaofeng 

3
C
Seasoned Practitioner
8 months ago
L
Seasoned
Lucas Carvalho de SousaSeasoned
asked in Developer and User Community

[FDM] How to create array of instance relationships via REST API?

Hello, I am trying to create a row instance in my data model that references multiple other types, via the REST API.I can do that fine with single relatioships, but when I get the ErrorCannot ingest into a connection property: supportingCast in view 'Film:Actor/xxxxxxxxxxx'.when I try an array of types like this:{ "externalId": "Oppenheimer", "space": "imdb", "lead": { "externalId": "CillianMurphy", "space": "imdb" }, "supportingCast": [ { "externalId": "EmilyBlunt", "space": "imdb" }, { "externalId": "RobertDowneyJr", "space": "imdb" } ]}The supportingCast property is the problem is this case.

5
J
Seasoned Practitioner
9 months ago
X
Seasoned
Xiaofeng WangSeasoned
asked in Developer and User Community

UI issue in FDM

Hi,We can resize the column width in the instance view in FDM, but if we scroll down to load more instances, the column width goes back the the default one. Could you help have a check if that is a bug in the UI?After resize to the externald column:After scroll down:Regards,Xiaofeng

1
J
Seasoned Practitioner
9 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

Retrieve dataframe from RAW table

I uploaded a CSV into a raw table in cognite data fusion. Now I have seen the raw table and see 200 numeric columns and 2 text columns are present. When i come to SDK and retrieve the rows from the same table as dataframe, it is showing 199 columns as numeric and 3 text columns. there was one specific column '0NA' which was originally a numeric column now shows the datatype as 'object' instead of 'float64'. I checked and there are no null values and no other NaN characters and it's just decimal values all across. When I fetch all numeric columns directly from CSV, I see the column as float-64 but after importing that CSV into RAW table, it is doing something else. I tried to see the unique values in the column and found these two as string,I suppose. In CSV , they were stored as number but when uploading to RAW table, it does convert into something like this. Please help if RAW table cannot accommodate values with more than 10E142 values such as these?. How to handle this scenario.'241

3
D
Practitioner
9 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

Delete root asset along with its children

I was trying to check SDK creating an asset hierarchy along with children (16). I created it in a improper structure and wanted to delete all of them and redo it from scratch. When i try deleting it, i am getting an error.   client.assets.delete(i.id)CogniteNotFoundError: Not found: [{'id': 1035393670500031}]The API Failed to process some items.But when i see the list for the parent id, I am to list all of the ids completelyassets_subtree = client.assets.list(asset_subtree_ids=[3043982824375333])for i in assets_subtree:    if i.name != 'LPM_YT_MODEL':        print(i.id,i.name,i.description) 

5
roman.chesnokov
Seasoned Practitioner
9 months ago
X
Seasoned
Xiaofeng WangSeasoned
asked in Developer and User Community

Limit on adding instance API call

Hi,Currently we have a limit on the POST models/instances where we only allow to post 1000 instances at a time.This operation is currently limited to 1000 nodes and/or edges at a time.Normally between the instances there are dependencies so it is not very easy to split the request to multiple calls. Do you think it is possible to lift that limit to a big number, f.ex. 10000? Regards,Xiaofeng

3
J
Seasoned Practitioner
9 months ago
L
Committed
Lihui MengCommitted
asked in Developer and User Community

Pagination and returning records at a specific page

Hello data modeling team, I tried the "first" and "after" filter on Query Explorer which enables us to retrieve the query result page by page, it works well.while we have an existing API which enables the consumer to specify an "offset" and a "limit" parameter. by the offset, the consumer could retrieve records at any specific page without the need to always go though from the beginning and then retrieve page by page, it's like the skip and limit syntax in mongodb, for example, db.myCollection.find().skip(200000).limit(50).we want to keep the API behavior so no modification needed for existing consumer, while we do not want to simulate the offset behavior by retrieving page by page within our own API since it will be very low efficient when there is a big dataset, so is it possible to provide the offset supporting in Cognite ? or any suggestions/methods we could try to implement this ? Thanks! 

2
J
Seasoned Practitioner
9 months ago
C
Active
carriechungActive
asked in Developer and User Community

FDM/Data modelling versioning management

Hi, I am looking for documentation regarding what kind of DM structure change is considered as safe change, breaking change or unsupported change in the data modelling but couldn’t find it.  And is it possible to provide best practices for DM versioning management? Kind advice.Thank you, Kind regards, Carrie 

1
J
Seasoned Practitioner
9 months ago
Anders Brakestad
Seasoned
Anders BrakestadSeasoned
asked in Developer and User Community

Sudden error in OAuth authentication

I have a helper function that returns a CogniteClient via interactive OAuth authentication. This worked literally yesterday, but gives me an error today.This is my function definition import osfrom cognite.client import CogniteClient, ClientConfigfrom cognite.client.credentials import OAuthInteractivefrom typing import Uniondef authenticate_to_cognite_oauth( project: str="akerbp", client_name: str="Employee", client_id: Union[str, type(None)]=None, tenant_id: Union[str, type(None)]=None, base_url: str="https://api.cognitedata.com" ) -> CogniteClient: """Authenticate to Cognite via OAuth interactive login. Return cognite.client.CogniteClient.""" if client_id is None: client_id = os.getenv("CDF_CLIENT_ID") if tenant_id is None: tenant_id = os.getenv("CDF_TENANT_ID") authority_url = f"https://login.microsoftonline.com/{tenant_id}" scopes = [f"{base_url}/.default"] creds = OAuthInteractive( authority_url=authority_u

3
Anders Brakestad
Seasoned
9 months ago
Vishnuvallabha Iyengar
Committed
Vishnuvallabha IyengarCommitted
asked in Developer and User Community

Guidance Needed: Implementing Time-Series Analysis with Cognite Python SDK

Hello Cognite Community,I'm currently in the process of deepening my understanding of the Cognite Python SDK, and I've encountered some hurdles that I believe this community could help me overcome. I'm working with a time-series dataset and aiming to use it to make machine learning-based predictions. To facilitate a more comprehensive response, I've provided a snapshot of the data I'm working with and detailed my queries below.Dataset Structure: The data comprises 5 columns: time, ws_E05, ws_E06, wp_true_E05, wp_true_E06. The 'ws_E05' represents the true wind speed at location E05, and 'wp_true_E06' signifies the true wind power at location E06.Query 1: I plan to utilize this dataset to forecast the subsequent 10 minutes using Linear Regression in the Python SDK on Cognite. Could you advise me if the present state of this time-series data is sufficient to get started, or are there certain modifications or preprocessing steps I should consider? I'm also interested in visualizing this da

2
Vishnuvallabha Iyengar
Committed
9 months ago
A
Committed
Amit RaneCommitted
asked in Developer and User Community

How to run Remote type extractor and give the configuration in extraction pipeline.

I want to pass the configuration and queries from the extractor pipeline to extractor (Cognite DB Extractor) which is I am running from my local but not able to start the extractor.Currently referring this doc: Configure extractors remotely | Cognite Documentationbut there is no sample for configuration file.or is there is way for passing queries for data extraction through extraction pipeline?

4
A
Committed
9 months ago
N
Active
Necula Simona-ElenaActive
asked in Developer and User Community

Loading CDF table to Power BI with errors

Hi,I need to connect to CDF with Power BI and when trying to load the table, I get this error:It only loads a limited rows of data (probably those rows have no errors) and I cannot change the format of the fields in PBI.I already whent thru the documentation , but the solution provided there does not work in this case (there is no option to change the data format).For other datasets (other data sources), where Power BI encounters errors, the data is still loaded (with errors), but in this case, the data is not loaded and the import is stopped by the error. This is also keeping for saving the report or using the data source.The behavior needed is that PowerBI is loading the entire dataset with errors and to change the field format before importing the data.Thank you!

6
C
Seasoned Practitioner
9 months ago
A
Seasoned
Adrian DamsSeasoned
asked in Developer and User Community

Configure an OpcUA Extractor to push a NodeSet to CDF

I am trying to configure an OpcUA extractor to push a NodeSet2.xml file to CDF. I have it basically setup and if I run it, there are no obvious errors, but I don’t see any kinds of assets being imported.  Am I expecting too much? Is the functionality in place to import an OpcUA Nodeset as Assets into my CDF project. If anyone has this working I would really appreciate a copy of the config file you are usingThanksAdrian

7
D
Committed
5 months ago
A
Committed
Amit RaneCommitted
asked in Developer and User Community

Creating timeseries external Ids by extractor

In the extractor I wanted to ingest the datapoints for the created timeseries in the given dataset.even though its ingesting datapoints for the created timeseries But also ingesting datapoints for timeseries which are not available or not created. It's also creating timeseries and ingesting datapoints.OR is there any way to add config or mapping for extractor??  

5
Dilini Fernando
Seasoned Practitioner
8 months ago

Badge winners

Show all badges
Terms & ConditionsCookie Settings