Community Discussions

Attn. @Jatin Sablok We found some issues with the Cognite pre-built OPC-UA extractor’s automatic re-connection to OPC server and to CDF as detailed in the test cases below.OPC-UA server reconnect test: OPC server is online and Cognite OPC-UA extractor is started. Then OPC server is restarted. Cognite extractor loses connection to OPC server, and is unable to re-connect automatically even after OPC server comes back online later. Extractor service has to be manually restarted to re-connect to OPC server.CDF reconnect test: Cognite OPC-UA extractor is started with network connection cable to CDF unplugged. Extractor log shows error messages. And extractor is unable to connect to CDF automatically even after network cable is plugged back in. Extractor service needs to be manually restarted to connect to CDF.Logs from both tests are attached. Could you please review them and let us know if these are known issues?

How to create IFSDB in events?

Since the backwards pagination is not supported and hasPreviosPage will remain false always. We are displaying page numbers and clicking on a number will fetch respective data. This is working fine when moving forward (next)We need solution for moving backwards in sequence or jumping to a certain page. Please suggest... 

Is there any way to define schedule for custom db extractor (based on cognite-extractor-utils)  alike default extractor. 

I have to do complex calculations and store the resulting data in the form of data frames (tabular form of data structure). The only way I see is to use the ‘sequences’ in CDF resource types. But I think CDF sequences doesn't allow to do data wrangling as we can do in pandas data frames. So, I wish to know if there is any best way to accomplish the storage of tabular data structures like data frames / arrays like what we can usually do in core Python. Basically, I wish to store data in structures like we typically have in core Python. Lists, Dataframes , arrays etc.  Any structure available in CDF?

Hi, what is required as access rights to be able to uplad a function ?  

Hi Team,In GraphQL there is no (known) option to filter null values of FDM View’s direct properties which refer to another view(s).For example: Consider the snapshot of the views.Here if we want to filter all MyTypeWrapper instances which has ‘myType’ property as null.How could it be achieved using GraphQL?-Mohit

I am unable to complete and proceed further, as  the below mentioned course as it is showing as  “Registered” even after completion Please give the solution even after clearing my system cache and tried in other browser as well https://learn.cognite.com/path/data-engineer-basics-transform-and-contextualize/match-entities-concept-and-ui  

Hi there!I have a usecase where a file is uploaded by a user to an API. The API then uploads the file to CDF Files. We want to avoid having to have the full file in memory at the same time, and therefore must stream the file contents from the request handler directly into CDF Files.There are two ways of achieving this:Stream the request body from the request handler directly into CDF Files’ upload URL Chunk the request body and upload each chunk as separate requests.The first option may be achievable, but I don’t believe the second option is possible.Do you have any insight whether it is possible to chunk a file upload like this in CDF Files?

Hi, We like to use Cognite AIR in one of our project .We got to know that it is getting decommissioned by end of 2023.Kindly confirm on that whether we should explore AIR now or we should not as it will not be available after this year. Thanks,  

I am trying to run a code to fetch timeseries based on some tags available in a project. While I execute the same code using jupyter notebooks in CDF online-notebook feature, the code runs fine. When I am trying to run the same code script in local after setting up connectivity using interactive-login and then when I run the timeseries retrieve code, I am getting an error. Please help.Code:from datetime import datetime, timezoneutc = timezone.utcpi= client.time_series.data.retrieve_dataframe(external_id=['pi:2FC1898.DACA.PV','pi:2TC1066.DACA.PV','pi:LAB_133-X013_APIGRAVOB','pi:2FC1898.PIDA.OP'], start=datetime(2023, 1, 1, tzinfo=utc), end=datetime(2023, 5, 1, tzinfo=utc), aggregates=["average"], granularity="1d")  Error- Traceback:---------------------------------------------------------------------------AttributeError Traceback (most recent call last)Cell In [9], line 5 1 from d

Hi, when I create a function that imports pandas I get  ModuleNotFoundError: No module named 'pandas'I expected that the function was hosted in a pre-configured runtime environment with many commonly used libraries already installed, including pandas.Any idea what I am doing wrong?

Hello! I have 3 quick questions that come to mind:From your perspective, what is the 1 sentence value statement of CDF?How can we justify all the manual work required to prepare the data to ingest into the platform?How do our customers save money by using CDF?

Hi ,  I’m trying to deploy an Azure function on Azure function app. But when I Included Cognite related libraries which I need to read and write data to Cognite data model. It was not working  even though I mentioned to include cognite in requirments.txt still not working. Has anybody else faced same issue ? Have you use azure function to connect to cognite (not the cognite functions)/

Hello I noticed that data type for table import thru CSV file is by default as string, how can I change it to other data type like boolean, array and etc?  Regards

We are using the online version of the Jupyter notebook from CDF portal for a client project - DEV and able to get the clientconfig/ client object and create and retrieve assets, run transformations, create datasets etc. Client IT team has created an app and registered in Azure and also shared the tenant ID, Client ID / name and secrets as well. When we use these parameters shared for this app and run the same code locally in a notebook, it is not able to perform certain tasks (such as data set creation etc.). Basically, the online version has all the IAM groups as {data engineer, data scientist Data Analyst, OIDC-Admin.}But when we set the configuration parameters client-ID, Tenant and secrets etc., we don't get the groups entirely as above but only comes as “Data Integration”.  This “Data-integration” has limited scope and doesn't allow to create datasets etc.  So how do we understand this part of roles and access management in CDF construct and applications registered in Azure AD? 

I have a time series data identified with TAGS and that can contain around 1500 to 5000+ records generated per day. I would have to perform a time weighted average and calculate the time-weighted value for the time-series data for the given times. How do I proceed to recreate the computation in Cognite since I got the PI data already sitting within CDF. Basically, got to recreate this function of OSI PI inside CDFPIAdvCalcDat(tagname, stime, etime, interval, mode, calcbasis, minpctgood, cfactor, outcode, PIServer) 

Hi team, In Hess team came up with one request from Documentum Extractor, please refer below context for the request. Currently we have Cognite connected to Documentum via the raw folder, which pulls in the raw file format from EDMS. We actually need to connect to Documentum's rendition folder where the PDF versions of all the files are saved. Cognite can only effectively contextualize PDF's, so we need to connect to the render folder directly. We contacted the documentum team, however they've said that only people on the Documentum team can connect to that folder. We just need the Cognite extractor to have access to that Rendition repository. Please let me know if you need more information. project - hess-dev, hess-us

I have a lot of timeseries objects in CDF datasets. I have a particular set of timeseries tags out of the innumerable list of timeseries objects (16 of them) and for each tag, there are a bunch of sub-tags (4 of them). So, in total, I will need to maintain a hierarchy of 16 tags and each having 4 tags and the overall total of 64 tags. I need to go and retrieve the datapoints for each of those 64 tags. So how do I store my desired list of timeseries tags along with their child tags within CDF. Where do I store them and maintain them. This list may be edited and needs flexibility to be edited based on business users need. IT is completely the enterprise choice.  Please share complete steps simulate them in CDF. This is actually to be done for yield-tracking analytics and all these tags corresponding to the yield groups/products in a refinery.  

I have csv files with asset information, asset hierarchy, time series objects and timeseries datapoints. I would like to load these to CDF and link to a dataset. Could someone walk me through this or refer me to some help files? 

i need support for the course Python SDK Transformations.Im running the comands from notebook that was given in git and a stoped on:result = client.transformations.run(asset_transformation.id, wait=False)The error resulted is :CogniteAPIError: Invalid source/destination credentials: Could not authenticate with the OIDC credentials. Please check your credentials. | code: 403 | X-Request-ID: 78e0e54a-a3ca-9bc4-b2f5-e303ed6a7207iv checked the authentication process and i can perform all task like search list add datasets assets and everything but i cant run the transformation.Any one knows how to solve it ?i have the same problen in the UI  

I have many thousand timeseries where I need to find the date of the first datapoints in each series. For each timeseries, I have no idea if the first datapoint is from this year, or from 20 years ago. Fetching data for several decades is not effective.  Any ideas how to get the first datapoint? Getting the last datapoint would also be great.  Thanks!

Executing transform from Raw/Staging to Dataset Events. Codeselect cast(`uniqueid` as BIGINT) as id, cast(`Start Time` as TIMESTAMP) as startTime, cast(`End Time` as TIMESTAMP) as endTime, 1626362640169782 as dataSetId, concat("Chem-Batch", "-", Unit, "-", `Batch ID`) as description, 'Process' as type, 'Batch' as subtype from `Chem-Batch`.`Batch-Unit`where `Start Time` > '2023-06-13'Preview shows expected 3 row results: Run yields this error:Request with id 90f5d941-7f0e-9553-9ca5-317d784434d1 to https://az-eastus-1.cognitedata.com/api/v1/projects/ra-istc-sandbox/events/update failed with status 400: Event id not found. Missing ids: [48, 49].It seems all columns data in Raw rows has appropriate data.Any ideas are welcome.Chris  

Hello Community,we have a option scheduling in cognite functions to run those functions periodically. But is there any way to trigger Cognite function when a particular event occur.thank you

A few quick questions to help with a potential client engagement focusing on generating insights from unstructured docs in file systems:Can we go beyond the 1MB limit to allow OCR, index and search of any free text within the documents. Is this configurable?   LLM/NLP Search: What level of fuzzy or NLP searches are supported on the indexed content of a document. For example could one search for a phrase that isn’t verbatim in the file, but close enough in meaning or spread out within a couple of places within the document.   Automation: I understand we can auto-extract file/folder names and even text from the initial portion of a document to create metadata. Can this metadata be used in an automated entity matching service to automatically relate (contextualize) a new document to the parent field, well, facility etc. Can the same automation work by purely relying on the text within the OCRed document to avoid the human having to categorize into folders and ensuring naming standards f