Community Discussions

Hi,Currently we have a limit on the POST models/instances where we only allow to post 1000 instances at a time.This operation is currently limited to 1000 nodes and/or edges at a time.Normally between the instances there are dependencies so it is not very easy to split the request to multiple calls. Do you think it is possible to lift that limit to a big number, f.ex. 10000? Regards,Xiaofeng

I uploaded a CSV into a raw table in cognite data fusion. Now I have seen the raw table and see 200 numeric columns and 2 text columns are present. When i come to SDK and retrieve the rows from the same table as dataframe, it is showing 199 columns as numeric and 3 text columns. there was one specific column '0NA' which was originally a numeric column now shows the datatype as 'object' instead of 'float64'. I checked and there are no null values and no other NaN characters and it's just decimal values all across. When I fetch all numeric columns directly from CSV, I see the column as float-64 but after importing that CSV into RAW table, it is doing something else. I tried to see the unique values in the column and found these two as string,I suppose. In CSV , they were stored as number but when uploading to RAW table, it does convert into something like this. Please help if RAW table cannot accommodate values with more than 10E142 values such as these?. How to handle this scenario.'241

Hi,we are looking for a 2D file/diagram viewer compatible with https://developer.cognite.com/api/#tag/Annotations to be used inside our own application, that is outside of the Cognite application suite.Question: What are the current plans regarding opening “Unified File Viewer” in a similar fashion as https://github.com/cognitedata/reveal?Bonus question: Is it / would it be possible to annotate other things than asset tags?

Hello,We are trying to fetch the edges of an view in a graphql.But graphql is returning only 50 edges at max. We see the cursor option in graphql, but it seems that it has been not implemented and is giving null even if there exists more data.So question is - When the cursor functionality for edges will be avaialble in graphql or some know issue as of now ? If this is a limitation how do we get edges data joined with the view data where the edge is present using the rest endpoints. Any best combination suggested as there will be multiple stages to resolve the cursor and the edges data, as we understand it.Request your quick ideas on this how it could be achieved.

I was trying to check SDK creating an asset hierarchy along with children (16). I created it in a improper structure and wanted to delete all of them and redo it from scratch. When i try deleting it, i am getting an error.   client.assets.delete(i.id)CogniteNotFoundError: Not found: [{'id': 1035393670500031}]The API Failed to process some items.But when i see the list for the parent id, I am to list all of the ids completelyassets_subtree = client.assets.list(asset_subtree_ids=[3043982824375333])for i in assets_subtree:    if i.name != 'LPM_YT_MODEL':        print(i.id,i.name,i.description) 

I have a daily feed data of the form coming as a file in sharepointDate AX1 AB1 AW1 AN1 AR1 AB1 BW1 BG1 BT1 01-01-2023 0.23 1.23 2.54 0.98 0.655 1.09 0.87 0.34 0.19 01-02-2023 0.34 0.56 0.44 0.89 0.576 0.81 1.29 1.11 0.67 …... ….. ….. …... …... …... …... …... …... …..  I need to pull this data and transform it into an applicable resource-type (Sequences in CDF). I don’t want to store as time series since I need this in tabular form to operate further and do matrix and dot product etc. Is there a way to write a transformation to push it as ‘sequences’ so that I can use it as a matrix to perform further calculations?Please share some example SQL transformation snippets to achieve this. 

Hello data modeling team, I tried the "first" and "after" filter on Query Explorer which enables us to retrieve the query result page by page, it works well.while we have an existing API which enables the consumer to specify an "offset" and a "limit" parameter. by the offset, the consumer could retrieve records at any specific page without the need to always go though from the beginning and then retrieve page by page, it's like the skip and limit syntax in mongodb, for example, db.myCollection.find().skip(200000).limit(50).we want to keep the API behavior so no modification needed for existing consumer, while we do not want to simulate the offset behavior by retrieving page by page within our own API since it will be very low efficient when there is a big dataset, so is it possible to provide the offset supporting in Cognite ? or any suggestions/methods we could try to implement this ? Thanks! 

Hi, I am looking for documentation regarding what kind of DM structure change is considered as safe change, breaking change or unsupported change in the data modelling but couldn’t find it.  And is it possible to provide best practices for DM versioning management? Kind advice.Thank you, Kind regards, Carrie 

I want to pass the configuration and queries from the extractor pipeline to extractor (Cognite DB Extractor) which is I am running from my local but not able to start the extractor.Currently referring this doc: Configure extractors remotely | Cognite Documentationbut there is no sample for configuration file.or is there is way for passing queries for data extraction through extraction pipeline?

Hi I have created a Cognite File Extractor from a local folder, and I want to create a log of all runs in a sub-folder. When changing the log level on the console and adding file as storage, nothing changes.  from config file: logger:  console:    level: DEBUG  file:    level: INFO    path: ".\\logs\\log.txt"  //also tried with full path    retention: 7 Are there some more steps to get the logger to change?I have run it both as local and admin. Hedda

I am running a transformation where in IN Clause I am giving values by selecting from mapping file in raw (which stores value as Item1, Item2)but not getting any desired results (giving 0 rows but not giving any error)e.gSELECT * from tableNamewhere columnName IN (SELECT propertyName from rawDB.mapping where columnName=Id)  

Hello Cognite Community,I'm currently in the process of deepening my understanding of the Cognite Python SDK, and I've encountered some hurdles that I believe this community could help me overcome. I'm working with a time-series dataset and aiming to use it to make machine learning-based predictions. To facilitate a more comprehensive response, I've provided a snapshot of the data I'm working with and detailed my queries below.Dataset Structure: The data comprises 5 columns: time, ws_E05, ws_E06, wp_true_E05, wp_true_E06. The 'ws_E05' represents the true wind speed at location E05, and 'wp_true_E06' signifies the true wind power at location E06.Query 1: I plan to utilize this dataset to forecast the subsequent 10 minutes using Linear Regression in the Python SDK on Cognite. Could you advise me if the present state of this time-series data is sufficient to get started, or are there certain modifications or preprocessing steps I should consider? I'm also interested in visualizing this da

Attn. @Jatin Sablok We found some issues with the Cognite pre-built OPC-UA extractor’s automatic re-connection to OPC server and to CDF as detailed in the test cases below.OPC-UA server reconnect test: OPC server is online and Cognite OPC-UA extractor is started. Then OPC server is restarted. Cognite extractor loses connection to OPC server, and is unable to re-connect automatically even after OPC server comes back online later. Extractor service has to be manually restarted to re-connect to OPC server.CDF reconnect test: Cognite OPC-UA extractor is started with network connection cable to CDF unplugged. Extractor log shows error messages. And extractor is unable to connect to CDF automatically even after network cable is plugged back in. Extractor service needs to be manually restarted to connect to CDF.Logs from both tests are attached. Could you please review them and let us know if these are known issues?

How to create IFSDB in events?

Since the backwards pagination is not supported and hasPreviosPage will remain false always. We are displaying page numbers and clicking on a number will fetch respective data. This is working fine when moving forward (next)We need solution for moving backwards in sequence or jumping to a certain page. Please suggest... 

Is there any way to define schedule for custom db extractor (based on cognite-extractor-utils)  alike default extractor. 

I have to do complex calculations and store the resulting data in the form of data frames (tabular form of data structure). The only way I see is to use the ‘sequences’ in CDF resource types. But I think CDF sequences doesn't allow to do data wrangling as we can do in pandas data frames. So, I wish to know if there is any best way to accomplish the storage of tabular data structures like data frames / arrays like what we can usually do in core Python. Basically, I wish to store data in structures like we typically have in core Python. Lists, Dataframes , arrays etc.  Any structure available in CDF?

Hi, what is required as access rights to be able to uplad a function ?  

Hi Team,In GraphQL there is no (known) option to filter null values of FDM View’s direct properties which refer to another view(s).For example: Consider the snapshot of the views.Here if we want to filter all MyTypeWrapper instances which has ‘myType’ property as null.How could it be achieved using GraphQL?-Mohit

I am unable to complete and proceed further, as  the below mentioned course as it is showing as  “Registered” even after completion Please give the solution even after clearing my system cache and tried in other browser as well https://learn.cognite.com/path/data-engineer-basics-transform-and-contextualize/match-entities-concept-and-ui  

Hi there!I have a usecase where a file is uploaded by a user to an API. The API then uploads the file to CDF Files. We want to avoid having to have the full file in memory at the same time, and therefore must stream the file contents from the request handler directly into CDF Files.There are two ways of achieving this:Stream the request body from the request handler directly into CDF Files’ upload URL Chunk the request body and upload each chunk as separate requests.The first option may be achievable, but I don’t believe the second option is possible.Do you have any insight whether it is possible to chunk a file upload like this in CDF Files?

Hi, We like to use Cognite AIR in one of our project .We got to know that it is getting decommissioned by end of 2023.Kindly confirm on that whether we should explore AIR now or we should not as it will not be available after this year. Thanks,  

I am trying to run a code to fetch timeseries based on some tags available in a project. While I execute the same code using jupyter notebooks in CDF online-notebook feature, the code runs fine. When I am trying to run the same code script in local after setting up connectivity using interactive-login and then when I run the timeseries retrieve code, I am getting an error. Please help.Code:from datetime import datetime, timezoneutc = timezone.utcpi= client.time_series.data.retrieve_dataframe(external_id=['pi:2FC1898.DACA.PV','pi:2TC1066.DACA.PV','pi:LAB_133-X013_APIGRAVOB','pi:2FC1898.PIDA.OP'], start=datetime(2023, 1, 1, tzinfo=utc), end=datetime(2023, 5, 1, tzinfo=utc), aggregates=["average"], granularity="1d")  Error- Traceback:---------------------------------------------------------------------------AttributeError Traceback (most recent call last)Cell In [9], line 5 1 from d

Hi, when I create a function that imports pandas I get  ModuleNotFoundError: No module named 'pandas'I expected that the function was hosted in a pre-configured runtime environment with many commonly used libraries already installed, including pandas.Any idea what I am doing wrong?

Hello! I have 3 quick questions that come to mind:From your perspective, what is the 1 sentence value statement of CDF?How can we justify all the manual work required to prepare the data to ingest into the platform?How do our customers save money by using CDF?