Developer and User Community

Join the discussions about delivering apps and data-driven decisions powered by Cognite Data Fusion. Click the + CREATE TOPIC button in the menu bar to start the conversation.

  • 552 Topics
  • 1,771 Replies

552 Topics

A
Active
andcondallActive
asked in Developer and User Community

Download openapi spec in CI

Trying to download the openapi spec over at https://api-docs.cognite.com/20230101/ by using the URL in the “download” link fails with 404 when trying to use the URL with wget/curl and “ADD” in a dockerfile. It seems like the link is generated behind the scenes during pageload by javascript. Is there a way to download the openapi spec in scripts e.g for use in CI pipelines and similar?

2
A
Active
19 hours ago
H
Hakim AREZKINew
asked in Developer and User Community

Testing CogniteAuthError with monkeypatch_cognite_client

Hello, is it possible to test the  CogniteAuthError using monkeypatch context manager  ?Regards ,Hakim Arezki 

0
H
21 hours ago
A
Active
andcondallActive
asked in Developer and User Community

Generate client from CDF openapi spec for Go

I am unable to generate client code for the CDF openapi spec at https://api-docs.cognite.com/20230101/ using deepmap/oapi-codegen for Go, seemingly because the spec contains some errors. Do anyone know of a way to generate an api-client in go for the CDF-api spec? Both commands below rely on having openapi spec downloaded to the current directory from https://api-docs.cognite.com/20230101/ Testing the spec for errors with redocly/cli: docker run --rm -it -v $PWD/swagger.json:/swagger.json redocly/cli lint /swagger.jsonFor me this results in “Validation failed with 25 errors and 382 warnings” Generating the client library using deepmap/oapi-codegen: go run github.com/deepmap/oapi-codegen/v2/cmd/oapi-codegen@v2.0.0 --config oapi-codegen-config.yaml swagger.jsonwith the this config file:package: "cdf_api"output: "cdf_api.gen.go"generate: models: true client: truecompatibility: circular-reference-limit: 100For me this results in the message “error generating code: error creating operat

0
A
Active
1 day ago
N
Committed
Neerajkumar BhatewaraSchlumberger
asked in Developer and User Community

OPC UA Extractor - is there a parameter to handle the source timezone similar to DB Extractor

Hello,In DB Extractor, I see a parameter that could be configured to specify how to treat the timezone parameter.  Is there something similar for OPC-UA Extractor as well ?If not how can the different timezone data could be handled or atleast Local timezone data ?

1
E
Practitioner
1 day ago
K
Active
Karguvel kalisekarActive
asked in Developer and User Community

Python SDK - List, Search, Query Instance from FDM

Hi Team,Is there any sample code available for retrieving a list of instance objects from FDM? I've reviewed the documentation's sample code, but I need additional clarification on where to provide Space, Object Type, Version, and Data Model.  Sample code available for Instance List. instance_list = c.data_modeling.instances.list(limit=5) I have also followed the steps outlined in the document URL below.. advancedListInstance queryContent searchInstances Thanks,Karguvel K

3
K
Active
1 day ago
G
grc5084New
asked in Developer and User Community

Issue Running Transformation to get data into CDF

Hello, I am going through Cognite Data Fusion Fundamentals and am having an issue that I cant figure out. After multiple attempts I am loading 1 row of data to try and figure out my issue.I make it to the “Preview the Transformation” step:  The next step is to “Run the Transformation” step and this is where I am running into an issue. I get the below error message and theres no suggestion on what to do to correct my apparent error. Any idea what to do or how to progress? I just want to finish this course. -George

2
D
Practitioner
1 day ago
R
rmawais791New
asked in Developer and User Community

CDF Data Points Benchmarks with OBDC and OPC UA to read data from other application

Hi I have an application to read data from and I have two options either to use ODBC or OPC UA directlyWhat are the benchmarks or limitations when it comes to aboveI have 30,000 data points to read with sample time of 1s and 5s and 10s 

2
D
Practitioner
1 day ago
R
Committed
Rajendra PasupuletiCommitted
asked in Developer and User Community

How to upload excel files to CDF files

How to upload excel files to CDF files, I am looking for ways to upload files programatically and schedule this using cron expression.  

7
D
Practitioner
1 day ago
D
Dexter NguyenNew
asked in Developer and User Community

Problem with ingesting datapoint immediately after Timeseries object creation with multiple concurrent jobs.

We’re attempting to execute 10 concurrent jobs, and each one will create approximately 20 Timeseries objects, immediately ingesting datapoints for these newly created Timeseries objects in a loop.However, if we try to run 500 similar jobs with 10 concurrently running at a time, about 10% of the jobs fail due to a ‘Timeseries not found’ error.We theorize that the creation of a Timeseries object can take some time. Therefore, immediately ingesting the datapoint might fail because Cognite doesn’t refresh quickly enough to find the corresponding Timeseries object. This issue only occurs when we attempt to run multiple concurrent jobs.Has anyone else encountered similar issues? 

4
D
Practitioner
1 day ago
Å
Årne ReimeNew
asked in Developer and User Community

Data model query for Bi-directional relationships

I have a question about querying a data model using the CDF SDK.In the Cognite SDK docs there is an example for retrieving actors in a movie (https://cognite-sdk-python.readthedocs-hosted.com/en/latest/data_modeling.html#query-instances) from the following data model: But if we would change this to finding which pumps belongs to a given facility from this data model (picture below), using a bi directional relationship between facility and pump, what changes must be made to the example from the sdk docs for this to work?  

2
D
Practitioner
1 day ago
P
Committed
Pedersen Jon-RobertCommitted
asked in Developer and User Community

PI Extractor and Extraction Pipelines

Hi, we are using the PI Extractor with Extraction Pipeline and is pushing the config through the pipeline.But we noticed that after updating Config in extraction pipeline we have to restart the PI Extractor  to force it to use the updated config.If we update the config we can se in run history that the extractor is reporting “Seen” on the new config revision but it is not loading and using the new config.Is there a way to force it to use new config?

7
D
Practitioner
1 day ago
G
grc5084New
asked in Developer and User Community

Uploading CSV to Staging Area

Hello,I have been trying to load a CSV file into a table while going through the Cognite Data Fusion Fundamentals course. I have tried for over an hour to load the data correctly. One of the column headers adds 2 commas after the text and is throwing off the SQL formula. I am using the stock CSV file provided in the course, loading it straight from my desktop and not making any changes to the file itself.   

4
D
Practitioner
1 day ago
G
Active
gghalebActive
asked in Developer and User Community

state for OPC-UA extractor

Hi There,How do you configure the state of OPC-UA extractor in the Raw layer?I know that you can do it through various types of state stores including InfluxDB or local, but I have a dataset and a table in the Staging area which I would like to get populated.Thanks in advance!

2
G
Active
1 day ago
Chris Selph
Seasoned
Chris SelphSeasoned
asked in Developer and User Community

Cognite DB Extractor 3.0.0 Stops After Execution

Running Cognite DB Extractor 3.0.0 as a Windows Service. Note that previous versions of this extractor have worked fine although some changes have been made to the config.yml since upgrading. Symptoms - According to the log, Upon starting the service, It seems the extractor will run and successfully query and upload (30 rows in this case) and then the service will STOP running immediately as indicated in the Windows Services app.There is an error in the log file about database/table already existing which does not make sense to me b/c usually errors will prevent the service from starting at all.  https://www.yamllint.com/ indicates the config.yml is valid. The only entry in Windows Application log is an “Informational” note that states:The description for Event ID 4098 from source DBExtractor cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer

5
Chris Selph
Seasoned
1 day ago
S
Shabnam OghbaieNew
asked in Developer and User Community

OPC UA extractor connection to CDF fails

I have provided the essential credentials within the config.yml file in the OpcUaExtractor config folder, however when run it get this error. How can I fix this?

1
D
Practitioner
4 days ago
K
Seasoned
Kristian NymoenSeasoned
asked in Developer and User Community

3D models, scoping to data sets

Hi. What is the recommended method for access control of 3D models? 3D capabilities can be scoped to data sets, but how can I assign a model to a particular data set, so that it is only accessible by a limited user group? Is it possible via the Python SDK or GUI?

2
Andre Alves
Seasoned
6 days ago
N
Committed
Neerajkumar BhatewaraSchlumberger
asked in Developer and User Community

SDK - Questions around Global Config

Hello,What are the best practices around setting the max-workers for a sdk. If I do not set the Global Config all the default values for workers and retries would be picked correct ? Can you give any reference on how we can set GlobalConfig, I mean I did not see any way to pass the GlobalConfig to ClientConfig ?

8
H
Practitioner
7 days ago
K
Seasoned
Kristian NymoenSeasoned
asked in Developer and User Community

Access control. Sharing data.

Hi. When setting up CDF we were recommended to use CDF data sets scoped to data sources to control access to data. However, we have use cases where we would like to share a selection of data from a source with external parties. What would be the recommended setup for this? Would we need build an API layer on top of CDF,  to copy data from a source dataset to multiple other datasets for sharing? Or could the security categories feature be used for this?The person in charge of sharing data in our case will often be a data owner / SME without much coding experience. Is there a way our data owners could move/copy a subset of data from a source dataset to a sharing dataset, or somehow label individual time series or assets in a manner making them accessible to specificaccess groups?

4
N
Practitioner
7 days ago
P
Seasoned
Peter QuinnSeasoned
asked in Developer and User Community

Is there a InfluxDB extractor?

We have customers using InfluxDB as a local on-prem storage mechanism. What’s the best extractor to use to push this data to CDF? I’ve heard OPC-UA is the way to do it, but what I find online are ways to store OPC-UA data into Influx, but not ways to get InfluxDB data into a OPC-UA server so I could use the OPC-UA extractor to get it into CDF.

4
P
Seasoned
8 days ago
E
Active
Espen JacobsenActive
asked in Developer and User Community

Retrieving many raw datapoints from many timeseries

Hi We have a web application that uses the cognite sdk to present timeseries and datapoints.We have two use cases:1. We have a case with 500 time series, and we need to retrieve the latest 1000 data points from the combined 500 time series.2. We have a case with 500 time series, and we need to retrieve all datapoints between two dates. For both cases we also need to know which timeseries each datapoint belongs to and we need raw values and can not use aggregates.What is the most effective approach to achieve this?

8
E
Active
8 days ago
H
Committed
Harshita SaiCommitted
asked in Developer and User Community

I would like to write unittest for Extractor run main function, Please help me to find mistakes in my code.

Hi Team,Please help me to find the mistakes in my code. Let me showcase the dummy codemain code:def main() -> None:    """    Main entrypoint    """    BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))    config_values_vault.set_vara()    with Extractor(        name="SAP_Extractor",        description="An extractor to extract asset hierarchy from SAP based on root node",        config_class=SapConfig,        # version=__version__,        # debugger        version="1.0.0",        run_handle=run,        metrics=metrics,        config_file_path= os.path.join(BASE_DIR, 'config.yaml'),    ) as extractor:        extractor.run() I have build code unittest for above code: def test_main():    with patch('os.path') as path_mock:        with patch.object(path_mock, 'abspath') as mock_abspath:            with patch.object(path_mock, 'dirname') as mock_dirname:                with patch.object(path_mock,'join') as mock_join:                    mock_abspath.return_value = '/p

5
Anita Hæhre
Seasoned Practitioner
9 days ago
R
Rimmi AnandNew
asked in Developer and User Community

Search in JSON object type column data

I am looking for function/query that can search in  JSON type data against array of values. Json can have nested keys. I tried using JSON_SEARCH or JSON_CONTAINS . getting Undefined function:JSON_SEARCH(data->'$[*]', JSON_ARRAY("abc")) Is there a way to search in JSON object keys for list of values ? for e.g :JSON col data {"name":"Back","address":{"location":{"country":[{"city":{"state":[‘jkl’,’fgh’]},"type":"home"},"empId":"e100043867","FullName":"Back"}JSON_ARRAY = [‘name’,’state’,’country’]check if JSON_ARRAY values exists in JSON data ?Any help would be appreciated.

3
D
Practitioner
9 days ago
R
Rimmi AnandNew
asked in Developer and User Community

Query to get column name from different table stores as column value

We have data model withtype abc{id: Stringkind: Stringname: Stringdescription: Stringkey: String}In other table , we have these columns stored as column valueskind attributes 1.0.0 name 1.0.0 description 1.0.0 key 1.0.0 name  need query to join on column value(stored as rows) from one table with solution model table data can group on by id or kind? As there can be n number of attributes, if we can do it dynamically to match column  value with column name.Any help would be appreciated

3
D
Practitioner
9 days ago
A
Committed
Adrian DamsCommitted
asked in Developer and User Community

Configure an OpcUA Extractor to push a NodeSet to CDF

I am trying to configure an OpcUA extractor to push a NodeSet2.xml file to CDF. I have it basically setup and if I run it, there are no obvious errors, but I don’t see any kinds of assets being imported.  Am I expecting too much? Is the functionality in place to import an OpcUA Nodeset as Assets into my CDF project. If anyone has this working I would really appreciate a copy of the config file you are usingThanksAdrian

6
E
Practitioner
9 days ago
R
rmawais791New
asked in Developer and User Community

CDF Data Points Benchmarks with OBDC and OPC UA

Hi I have an application to read data from and I have two options either to use ODBC or OPC UA directlyWhat are the benchmarks or limitations when it comes to aboveI have 30,000 data points to read with sample time of 1s and 5s and 10s 

1
D
Practitioner
10 days ago

Badge winners

  • Cognite Data Engineer Basics - T&C
    sumitgawalihas earned the badge Cognite Data Engineer Basics - T&C
  • Cognite Data Engineer Basics - Integrate
    sumitgawalihas earned the badge Cognite Data Engineer Basics - Integrate
  • Cognite Domain Expert Basics
    Mitch Rashhas earned the badge Cognite Domain Expert Basics
  • Cognite Domain Expert Basics
    Rajeshhas earned the badge Cognite Domain Expert Basics
  • Cognite Data Scientist Basics
    Abdulmohsen Alateeqhas earned the badge Cognite Data Scientist Basics
Show all badges
Terms & ConditionsCookie Settings