Developer and User Community

Join the discussions about delivering apps and data-driven decisions powered by Cognite Data Fusion. Click the + CREATE TOPIC button in the menu bar to start the conversation.

  • 701 Topics
  • 2,181 Replies

701 Topics

C
Active
carriechungActive
asked in Developer and User Community

FDM/Data modelling versioning management

Hi, I am looking for documentation regarding what kind of DM structure change is considered as safe change, breaking change or unsupported change in the data modelling but couldn’t find it.  And is it possible to provide best practices for DM versioning management? Kind advice.Thank you, Kind regards, Carrie 

1
J
Seasoned Practitioner
9 months ago
A
Committed
Amit RaneCommitted
asked in Developer and User Community

How to run Remote type extractor and give the configuration in extraction pipeline.

I want to pass the configuration and queries from the extractor pipeline to extractor (Cognite DB Extractor) which is I am running from my local but not able to start the extractor.Currently referring this doc: Configure extractors remotely | Cognite Documentationbut there is no sample for configuration file.or is there is way for passing queries for data extraction through extraction pipeline?

4
A
Committed
9 months ago
N
Active
NonstadActive
asked in Developer and User Community

Logging of Cognite File Extractor

Hi I have created a Cognite File Extractor from a local folder, and I want to create a log of all runs in a sub-folder. When changing the log level on the console and adding file as storage, nothing changes.  from config file: logger:  console:    level: DEBUG  file:    level: INFO    path: ".\\logs\\log.txt"  //also tried with full path    retention: 7 Are there some more steps to get the logger to change?I have run it both as local and admin. Hedda

4
C
Seasoned Practitioner
9 months ago
A
Committed
Amit RaneCommitted
asked in Developer and User Community

IN clause not working as expected in tranformations

I am running a transformation where in IN Clause I am giving values by selecting from mapping file in raw (which stores value as Item1, Item2)but not getting any desired results (giving 0 rows but not giving any error)e.gSELECT * from tableNamewhere columnName IN (SELECT propertyName from rawDB.mapping where columnName=Id)  

5
S
Practitioner
9 months ago
Vishnuvallabha Iyengar
Committed
Vishnuvallabha IyengarCommitted
asked in Developer and User Community

Guidance Needed: Implementing Time-Series Analysis with Cognite Python SDK

Hello Cognite Community,I'm currently in the process of deepening my understanding of the Cognite Python SDK, and I've encountered some hurdles that I believe this community could help me overcome. I'm working with a time-series dataset and aiming to use it to make machine learning-based predictions. To facilitate a more comprehensive response, I've provided a snapshot of the data I'm working with and detailed my queries below.Dataset Structure: The data comprises 5 columns: time, ws_E05, ws_E06, wp_true_E05, wp_true_E06. The 'ws_E05' represents the true wind speed at location E05, and 'wp_true_E06' signifies the true wind power at location E06.Query 1: I plan to utilize this dataset to forecast the subsequent 10 minutes using Linear Regression in the Python SDK on Cognite. Could you advise me if the present state of this time-series data is sufficient to get started, or are there certain modifications or preprocessing steps I should consider? I'm also interested in visualizing this da

2
Vishnuvallabha Iyengar
Committed
9 months ago
A
Committed
Anand Sankar AananthakrishnanCommitted
asked in Developer and User Community

OPC-UA Extractor Issues

Attn. @Jatin Sablok We found some issues with the Cognite pre-built OPC-UA extractor’s automatic re-connection to OPC server and to CDF as detailed in the test cases below.OPC-UA server reconnect test: OPC server is online and Cognite OPC-UA extractor is started. Then OPC server is restarted. Cognite extractor loses connection to OPC server, and is unable to re-connect automatically even after OPC server comes back online later. Extractor service has to be manually restarted to re-connect to OPC server.CDF reconnect test: Cognite OPC-UA extractor is started with network connection cable to CDF unplugged. Extractor log shows error messages. And extractor is unable to connect to CDF automatically even after network cable is plugged back in. Extractor service needs to be manually restarted to connect to CDF.Logs from both tests are attached. Could you please review them and let us know if these are known issues?

6
A
Committed
9 months ago
Sumit Bondlewad
Committed
Sumit BondlewadCommitted
asked in Developer and User Community

How to create IFSDB in events?

How to create IFSDB in events?

1
E
Seasoned Practitioner
10 months ago
M
Active
Mohammad ImranActive
asked in Developer and User Community

Backwards Pagination not supported

Since the backwards pagination is not supported and hasPreviosPage will remain false always. We are displaying page numbers and clicking on a number will fetch respective data. This is working fine when moving forward (next)We need solution for moving backwards in sequence or jumping to a certain page. Please suggest... 

3
J
Seasoned Practitioner
10 months ago
M
Committed
Mohit ShakalyCommitted
asked in Developer and User Community

Schedule for custom db extractor

Is there any way to define schedule for custom db extractor (based on cognite-extractor-utils)  alike default extractor. 

2
M
Committed
10 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

Derived data entities

I have to do complex calculations and store the resulting data in the form of data frames (tabular form of data structure). The only way I see is to use the ‘sequences’ in CDF resource types. But I think CDF sequences doesn't allow to do data wrangling as we can do in pandas data frames. So, I wish to know if there is any best way to accomplish the storage of tabular data structures like data frames / arrays like what we can usually do in core Python. Basically, I wish to store data in structures like we typically have in core Python. Lists, Dataframes , arrays etc.  Any structure available in CDF?

4
E
Practitioner
10 months ago
C
Active
czachActive
asked in Developer and User Community

Upload function

Hi, what is required as access rights to be able to uplad a function ?  

1
S
Practitioner
10 months ago
M
Committed
Mohit ShakalyCommitted
asked in Developer and User Community

How to filter null values of FDM view's inherent property in GraphQL

Hi Team,In GraphQL there is no (known) option to filter null values of FDM View’s direct properties which refer to another view(s).For example: Consider the snapshot of the views.Here if we want to filter all MyTypeWrapper instances which has ‘myType’ property as null.How could it be achieved using GraphQL?-Mohit

3
M
Committed
10 months ago
Y
Active
yeshwanthActive
asked in Developer and User Community

match Entities :concept and ui -unable to complete the course

I am unable to complete and proceed further, as  the below mentioned course as it is showing as  “Registered” even after completion Please give the solution even after clearing my system cache and tried in other browser as well https://learn.cognite.com/path/data-engineer-basics-transform-and-contextualize/match-entities-concept-and-ui  

2
Y
Active
10 months ago
Eric Stein-Beldring
Seasoned Practitioner
Eric Stein-Beldring
posted in Developer and User Community

New webinar series coming soon – what use cases should we solve?

Hello Cognite Hub Community!We’re working on a new webinar series where we focus on concrete use case and solve it from start to finish with Cognite Data Fusion. We’d like to hear from you – which specific industrial use cases would you like us to solve? Please give us as much detail and context as you can.Do you have a use case solution you’d be interested in co-presenting with us? Fantastic. Please reach out to me directly on Cognite Hub or by email (eric.stein@cognite.com) and I’ll be in touch.Thanks everyone and looking forward to hearing from you!

0
Eric Stein-Beldring
Seasoned Practitioner
10 months ago
thomafred
Seasoned
thomafredSeasoned
asked in Developer and User Community

Streaming upload to CDF Files

Hi there!I have a usecase where a file is uploaded by a user to an API. The API then uploads the file to CDF Files. We want to avoid having to have the full file in memory at the same time, and therefore must stream the file contents from the request handler directly into CDF Files.There are two ways of achieving this:Stream the request body from the request handler directly into CDF Files’ upload URL Chunk the request body and upload each chunk as separate requests.The first option may be achievable, but I don’t believe the second option is possible.Do you have any insight whether it is possible to chunk a file upload like this in CDF Files?

10
C
Seasoned Practitioner
10 months ago
N
Committed
nabanitakarCommitted
asked in Developer and User Community

Is Cognite AIR getting decommissioned.

Hi, We like to use Cognite AIR in one of our project .We got to know that it is getting decommissioned by end of 2023.Kindly confirm on that whether we should explore AIR now or we should not as it will not be available after this year. Thanks,  

8
Arun Arunachalam
Seasoned Practitioner
10 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

cognite-sdk

I am trying to run a code to fetch timeseries based on some tags available in a project. While I execute the same code using jupyter notebooks in CDF online-notebook feature, the code runs fine. When I am trying to run the same code script in local after setting up connectivity using interactive-login and then when I run the timeseries retrieve code, I am getting an error. Please help.Code:from datetime import datetime, timezoneutc = timezone.utcpi= client.time_series.data.retrieve_dataframe(external_id=['pi:2FC1898.DACA.PV','pi:2TC1066.DACA.PV','pi:LAB_133-X013_APIGRAVOB','pi:2FC1898.PIDA.OP'], start=datetime(2023, 1, 1, tzinfo=utc), end=datetime(2023, 5, 1, tzinfo=utc), aggregates=["average"], granularity="1d")  Error- Traceback:---------------------------------------------------------------------------AttributeError Traceback (most recent call last)Cell In [9], line 5 1 from d

4
roman.chesnokov
Seasoned Practitioner
10 months ago
C
Active
czachActive
asked in Developer and User Community

Cognite function with pandas

Hi, when I create a function that imports pandas I get  ModuleNotFoundError: No module named 'pandas'I expected that the function was hosted in a pre-configured runtime environment with many commonly used libraries already installed, including pandas.Any idea what I am doing wrong?

1
C
Practitioner
10 months ago
D
Active
Derek MeeseActive
asked in Developer and User Community

Initial 3 Questions

Hello! I have 3 quick questions that come to mind:From your perspective, what is the 1 sentence value statement of CDF?How can we justify all the manual work required to prepare the data to ingest into the platform?How do our customers save money by using CDF?

4
Dilini Fernando
Seasoned Practitioner
10 months ago
G
Active
Gagan GuptaActive
asked in Developer and User Community

including cognite libraries for Azure function app not working

Hi ,  I’m trying to deploy an Azure function on Azure function app. But when I Included Cognite related libraries which I need to read and write data to Cognite data model. It was not working  even though I mentioned to include cognite in requirments.txt still not working. Has anybody else faced same issue ? Have you use azure function to connect to cognite (not the cognite functions)/

4
G
Active
10 months ago
H
Active
Haw Keat LimActive
asked in Developer and User Community

Data Type in table which import thru CSV file

Hello I noticed that data type for table import thru CSV file is by default as string, how can I change it to other data type like boolean, array and etc?  Regards

2
Dilini Fernando
Seasoned Practitioner
10 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

clientconfig using client

We are using the online version of the Jupyter notebook from CDF portal for a client project - DEV and able to get the clientconfig/ client object and create and retrieve assets, run transformations, create datasets etc. Client IT team has created an app and registered in Azure and also shared the tenant ID, Client ID / name and secrets as well. When we use these parameters shared for this app and run the same code locally in a notebook, it is not able to perform certain tasks (such as data set creation etc.). Basically, the online version has all the IAM groups as {data engineer, data scientist Data Analyst, OIDC-Admin.}But when we set the configuration parameters client-ID, Tenant and secrets etc., we don't get the groups entirely as above but only comes as “Data Integration”.  This “Data-integration” has limited scope and doesn't allow to create datasets etc.  So how do we understand this part of roles and access management in CDF construct and applications registered in Azure AD? 

6
Dilini Fernando
Seasoned Practitioner
10 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

time series data - calculate time weighted average

I have a time series data identified with TAGS and that can contain around 1500 to 5000+ records generated per day. I would have to perform a time weighted average and calculate the time-weighted value for the time-series data for the given times. How do I proceed to recreate the computation in Cognite since I got the PI data already sitting within CDF. Basically, got to recreate this function of OSI PI inside CDFPIAdvCalcDat(tagname, stime, etime, interval, mode, calcbasis, minpctgood, cfactor, outcode, PIServer) 

3
Dilini Fernando
Seasoned Practitioner
10 months ago
M
Active
muthubabuhActive
asked in Developer and User Community

Documentum Extractor - Issues and enhancement requests

Hi team, In Hess team came up with one request from Documentum Extractor, please refer below context for the request. Currently we have Cognite connected to Documentum via the raw folder, which pulls in the raw file format from EDMS. We actually need to connect to Documentum's rendition folder where the PDF versions of all the files are saved. Cognite can only effectively contextualize PDF's, so we need to connect to the render folder directly. We contacted the documentum team, however they've said that only people on the Documentum team can connect to that folder. We just need the Cognite extractor to have access to that Rendition repository. Please let me know if you need more information. project - hess-dev, hess-us

5
Dilini Fernando
Seasoned Practitioner
10 months ago
E
Practitioner
eashwar11
asked in Developer and User Community

Timeseries tags

I have a lot of timeseries objects in CDF datasets. I have a particular set of timeseries tags out of the innumerable list of timeseries objects (16 of them) and for each tag, there are a bunch of sub-tags (4 of them). So, in total, I will need to maintain a hierarchy of 16 tags and each having 4 tags and the overall total of 64 tags. I need to go and retrieve the datapoints for each of those 64 tags. So how do I store my desired list of timeseries tags along with their child tags within CDF. Where do I store them and maintain them. This list may be edited and needs flexibility to be edited based on business users need. IT is completely the enterprise choice.  Please share complete steps simulate them in CDF. This is actually to be done for yield-tracking analytics and all these tags corresponding to the yield groups/products in a refinery.  

3
roman.chesnokov
Seasoned Practitioner
10 months ago

Badge winners

Show all badges
Terms & ConditionsCookie Settings