Developer and User Community

Join the discussions about delivering apps and data-driven decisions powered by Cognite Data Fusion. Click the + CREATE TOPIC button in the menu bar to start the conversation.

  • 689 Topics
  • 2,166 Replies

689 Topics

Ben Brandt
Seasoned
Ben BrandtSeasoned
asked in Developer and User Community

DirectQuery For Future Datapoints

Hello!I am querying for DataPoints using the PostgreSQL Gateway, which I assume will have the same behavior as the DirectQuery feature built on the PostgreSQL Gateway.  What I am noticing is that I am not able to retrieve any DataPoints for future dates.  Is this intended or a bug?  Most of the data we produce and manage is in the future, so this would be an important feature for us if it could be supported.  Thanks! Query results returned in Azure Data Studio from PostgreSQL gateway:In CDF these time series have datapoints representing predictions far out into the future: 

2
Ben Brandt
Seasoned
1 year ago
K
Seasoned
Kristian NymoenSeasoned
asked in Developer and User Community

Problem with colon exporting charts data as CSV on windows

Hi.Would it be possible to change the name format of the CSV file downloaded from cognite charts? The colon in “New chart 2023-02-22 15:43:59 (timeseries).csv” makes it difficult to unzip the file on Windows.

2
K
Seasoned Practitioner
1 year ago
gedi
Seasoned Practitioner
gediSeasoned Practitioner
posted in Developer and User Community

User survey - tell us what FDM is for you

Hi, there! At Cognite, our customers are at the center of everything we do. Our goal is to help you to solve the industrial data problems daily. It's all about you & your industrial data.  To make sure we're giving you the best experience possible with the Flexible Data Model (FDM), we'd like to learn about your FDM expectations. Please spend a few minutes to share your thoughts with us in this short survey. The information you give will be confident, so be honest. We really want to know what you think. The survey 👉  https://forms.gle/d8BzpW4jmG4YN9kSA Thanks for your help! Cheers,On behalf of the FDM team   

2
gedi
Seasoned Practitioner
1 year ago
I
Active
ikequarshActive
asked in Developer and User Community

Generate a personal access token - Using Databricks Community edition

Is there anyway, one can generate a personal access token using Databricks community Edition? I am working through the training and the only environment I have is to use the community edition. The training requires “Generate a personal access token” which is not possible. Is there any workaround? 

2
Madina Kholmatova
Seasoned Practitioner
1 year ago
Ben Brandt
Seasoned
Ben BrandtSeasoned
asked in Developer and User Community

Document Behavior of Create Time Series Status Code 422

Please add better documentation around the behavior of status code 422 for Create Time Series.  Specifically, I’d like these questions answered:When multiple external ids are given and some are duplicate and some are not, will the time series that are not duplicates be inserted and will these created ids be returned in the response? Does this function similar to datapoints where create is actually create or update and the duplicates are updated with the data provided?  If not, what is the recommended pattern for create or update of time series entities through the API?I am aware of advice here around “EAFP vs LBYL”, but I am in a case where 98% of the time the time series will not exist and I’d like to optimize for this case.  Thanks!

2
C
Seasoned Practitioner
1 year ago
Jo Øvstaas
Seasoned
Jo ØvstaasSeasoned
asked in Developer and User Community

Does FDM support RDF, SHACL and SPARQL

Hi from HUB Ocean and the Ocean Data Platform. We are participating in the open ocean data cataloging scene, for instance through the EU ILIAD “Digital twin of the ocean” project.Home | Iliad - Digital Twin of the Ocean (ocean-twin.eu)In order to improve the interoperability between the myriad of ocean data portals, web services and data catalogues out there, it will be very important for the Ocean Data Platform data catalog to support technologies and concepts like RDF (Resource Description Framework), SHACL (Shapes Constraint Language) and SPARQL (RDF query language). Is this already supported by FDM or is it on the roadmap?

2
Jo Øvstaas
Seasoned
1 year ago
L
Active
lmossActive
asked in Developer and User Community

PID Contextualization Errors

I just finished the cognite academy examples on contextualization. I did notice on the PID contextualization example that there were quite a few errors in what seems potentially character recognition pipeline to identify tags in the PDF.The tutorial stated that “these were all good” and “we can accept all”. I suspect such a process would create missing or strange links in the contextualized dataset.Are these known issues?I lack a bit the understanding of the context for the importance of these mispredictions,but I thought to report them anyway just in case.Happy to support you on improving these if they are something that needs improvement.

2
C
Seasoned Practitioner
1 year ago
Juan Alvarado
Seasoned
Juan AlvaradoSeasoned
asked in Developer and User Community

recommendations for managing/decomposing complex models

As we develop richer models supporting a greater variety of workflows, we have started asking questions about model maintenance and modularization. Decomposing large models into sub-models that have low coupling seems attractive, but would require some type of referential support across models. Is Cognite considering such a feature, or how do we envision managing complex models in general? As you are probably aware large models have an increased risk that a breaking change in a peripheral type will force a new version on the entire model. Since data must be manually migrated today, this creates a maintenance load that we would be keen to address.

2
Anders Hafreager
Practitioner
1 year ago
E
Active
Erik PerssonActive
asked in Developer and User Community

OData service capabilities

We are interested in retrieving data from cdf through the odata service. We would do this not from Power BI, but from another Power Platform component, namely Dataverse. (the purpose would be to create "virtual tables" there which we could query, and in that way we could control access to the data more granularly then we are able to thorough the Cognite API) The documentation we find only mentions the OData connector in relation to Power BI (e.g. Cognite Power BI connector | Cognite documentation (cogniteapp.com) and Tips and best practices | Cognite Documentation). Is Cognite's OData service specifically tailored for Power BI, or is it a full implementation according to the OData standard? Is anyone able to recommend for or against attempting the proposed approach?In case the service implements the standard, does anyone know which version of the standard is it?

2
E
Active
1 year ago
S
Committed
simenriCommitted
asked in Developer and User Community

Charts: Possible bug in "constant value"

Hi, In my endeavor to make a percentwise threshold in Charts: I.e. how many percent of the last week was a time series above a certain value, I stumbled on what I think is a bug.I need to divide the time the timeseries is above a threshold by the total duration of the time window. I therefore tried the “constant value”-function, and set it to one. The default granularity is then at 7 days. I can change the granularity to 1 day as in the screenshot below:  However, when I change the granularity below this. Eg. 1 hour, I get the following warning: Ideally I want it to 1m or something like that, so that I have sufficient granularity to measure percentage above threshold over  the course of a day.Any suggestions as to how I can fix this, or an alternative way to reach my goal?Thank you

2
Eric Stein-Beldring
Seasoned Practitioner
1 year ago
Adam Gunther
Active
Adam GuntherActive
asked in Developer and User Community

Cognite ML: Data Modeling Hands-on Lab

I am attempting to run the python file associated with the Data Modeling hands-on lab (https://colab.research.google.com/drive/11NdrGUzFhbwsOlKqXVfYRqu9s1DzAw4b?usp=sharing). I am receiving the same error every time the ‘plot’ function is called: “TypeError: Cannot interpret '<attribute 'dtype' of 'numpy.generic' objects>' as a data type”. After some debugging, the data seems to be in a dataframe data type. I suspect there is some version discrepancy between numpy and pandas, has anyone else seen this error? Thank you, Adam

2
V
Practitioner
1 year ago
Pierre Pernot
Practitioner
Pierre PernotPractitioner
published in Developer and User Community

Cognite Replicator

A common setup that we have at Cognite is having 3 different environments: development, test and production. We have one specific Cognite Data Fusion (CDF) project for each of these environments. This setup allows to develop new features without interfering with production data. Also, it makes the deployment of new features to production much safer, since they are tested before.Sometimes, you still want to have the data you used during your tests, in your production environment. For example, if you extracted some data from sensors and saved them as time series in CDF, you may want to keep this historical data when you move to production. Same goes with files, assets, datasets, functions etc. Another example could be the replication of the data from a full production CDF project, to one of your customer’s project, containing only a subset of the source one.At Cognite, we have a python package (available at https://pypi.org/project/cognite-replicator/) whose purpose is to copy the select

962
Pierre Pernot
Practitioner
1 year ago
S
Practitioner
Satoshi YamadaPractitioner
asked in Developer and User Community

Charts: What is the easiest way to configure float precision?

Hi community!I got a query from our customer that they’d like to control the float precision in Chart.I found “Round” function in “Operators” and confirmed applying that indsl to the timeseries data does work.But I’d like to ask if there is any other, easier, way because creating function per every timeseries data is a bit boring.Thanks!

2
S
Practitioner
1 year ago
N
Committed
Niranjan Madhukar KarvekarCommitted
asked in Developer and User Community

Transformation to RAW

Hi,I have data in raw table in this format:Table1:id p1 p2 p3 1 2.0 3.0 4.0 2 10.0 15.0 20.0  I wanted to transform this details into following format through transformation id property value 1 p1 2.0 1 p2 3.0 1 p3 4.0 2 p1 10.0 2 p2 15.0 2 p3 20.0   so I have written following transformation query:SELECT id, property, valueFROM `db1`.`Table1`UNPIVOT (value FOR property IN (p1, p2,p3)) UP This query works in SQL studio, but in COGNITE transformation UI , I see below error:Mismatched input 'FOR' expecting {')', ',', '-'}(line 3, pos 15) Can you please help to understand what is missing in this?  

2
N
Committed
1 year ago
Anders Brakestad
Seasoned
Anders BrakestadSeasoned
asked in Developer and User Community

cognite-sdk for Python as a conda package

As of today I think the Python SDK is only accessible via pip. While I can install a pip package into a conda environment, I don’t really like this solution. Are you planning on releasing a conda package for the Python SDK?

2
H
Practitioner
3 months ago
K
Seasoned
Kristian NymoenSeasoned
asked in Developer and User Community

Aggregates of synthetic timeseries on templates / datapointsWithGranularity

Hi. How can I get aggregated values from synthetic timeseries fields on templates? Intuitively, I would expect the “datapointsWithGranularity” operator in the graphQL query to give me this, but I get the same results as for the “datapoints” operator. Should the aggregation type and granularity be part of the definition of the synthetic timeseries field in the template? If so, how? 

2
E
Practitioner
1 year ago
S
Committed
simenriCommitted
asked in Developer and User Community

AIR: Bug when choosing custom time-span

Hi, In the AIR application, one can choose to see the relevant time series for 5 years, 1 year, 6 months, 1 month, and “Custom”. When clicking “Custom”, a date choosing panel shows up. When selecting the end date, the application freezes, and one has to refresh the application to get it back. (Feilkode: RESULT_CODE_HUNG)  

2
A
Practitioner
1 year ago
C
Active
CLKRL2Active
asked in Developer and User Community

Remote App - Time Series trend window, cursor detail items window, tag name too short, value has > 10 decimal places.

I was looking at trends in the Remote App web version, using the cursor to touch tags. The detail pane shows the tag name(s) and current value where the cursor x axis crosses. The issue is the tag names are cutoff and mostly unreadable, the tag value is showing almost 10 or more decimal places. I would like to see the whole tag name and 1 or 2 decimal places, or use the signficant digits from the PI Server in the first place.

2
Eric Stein-Beldring
Seasoned Practitioner
1 year ago
Sverre Dorheim
Committed
Sverre DorheimCommitted
posted in Developer and User Community

Manage data governance at scale in CDF

WhyWhen starting out in Cognite Data Fusion (CDF) project, it's natural to start by creating data governance elements like groups, datasets, and Raw databases from the CDF user interface. But as the solution begins to scale, you'll quickly realize that it is demanding to set up a detailed configuration handling multiple solutions, sources, roles, and other dimensions. For scaling, precise control is needed for access management and data governance and to enforce the guidelines and rules across the solution. The problem is not trivial, and a good way to solve it is by replacing the manual approach with a configuration-driven system, where the configuration language supports higher-level concepts for data lineage and access control. With configuration files as the foundation, you can set up an automated DevOps process and review and approve any changes to the structure before they are deployed. This approach also dramatically simplifies sharing the same configuration across multiple envi

2
K
Practitioner
1 year ago
M
Committed
Mitul Nikunjkumar GandhiCommitted
asked in Developer and User Community

Synthetic timeseries API does not support "sum" aggregation

In grafana, we want to display total of all timeseries data-points for all assets connected to particular asset for which we are passing query like this:ts{assetSubtreeIds=[{id=$Org4}], name='oil-production-rate'}where value of $Org1 is coming from dashboard variable.and since cognite grafana plugin does not handle cursor api responses and synthetic timeseries api have some limitation in returning 10k records only in a request, to reduce number of data-points, we are trying to use aggregation: ‘sum’ and granularity: ‘30d’ since there could be more than 10k data-points. But its is throwing error saying “sum” is not valid aggregation type of synthetic timeseries. Any reason why “sum” aggregation is not supported for synthetic timeseries.(screenshot attached)

2
S
Practitioner
1 year ago
R
Active
Revathi.rameshActive
asked in Developer and User Community

Unable to authenticate to query datasets from python SDK

Hi, I am currently doing my Data Engineer Basics - “Learn to Use the Cognite Python SDK “ Course. while executing the notebooks, getting below error with the first “List” notebook even after successful Cognite client authentication.CogniteAPIError: Resource not found. This may also be due to insufficient access rights. | code: 403 | X-Request-ID: 52b3d730-caea-9d32-bfc4-649041e43c3e Note: I am not using Poetry as I am not getting option to change Kernel in my Jupyter.

2
R
Active
1 year ago
M
Active
maryamgholamiActive
posted in Developer and User Community

using map and distinct in query

Hi, I am trying to transform data from Raw to timeseries in CDF. I made a query and everything seems ok, but when I use distinct to check the external id uniqueness, I get error message:"Cannot have map type columns in DataFrame which calls set operations(intersect, except, etc.), but the type of column metadata is map<string,string>; I don’t know why I get this message, Isn’t it possible to use distinct when wi do MAP or To_Metadata mapping? 

2
M
Active
1 year ago
I
Seasoned
ibrahim.alsyedSeasoned
asked in Developer and User Community

COGNITE REVEAL SDK: POINT CLOUD AND 3D

I realizd that when leveraging Cognite REVEAL in application development, can we not have Point Cloud/360 Images annd 3D in one REVEAL? Why do i have to select which one? is that not embedded within the same where i can turn on or off the Point Cloud or 360 Images?

2
C
Practitioner
1 year ago
K
Committed
Kumar Varaganti SharathCommitted
asked in Developer and User Community

"Appears in" field showing Incorrect count for Asset

In CDF, when we click on any Asset , there is a file tab, where it shows Appears In and Linked files for that asset.But the Count is not matching with list of files its showing for Appears In.In the below screenshot, the appears in count showing as 20. But only 16 files were listed. 

2
K
Committed
1 year ago
K
Committed
Kumar Varaganti SharathCommitted
asked in Developer and User Community

AssetID not getting removed from file when you delete any annotations/boundaries on a file

Asset ID not getting removed from file  when you delete any annotations/boundaries on a file.when you manually delete any contextualization result on a file, only the annotations are getting deleted , but AssetId’s linked to that file still exists.  

2
A
Practitioner
1 year ago

Badge winners

Show all badges
Terms & ConditionsCookie Settings