Developer Community

I created an asset hierarchy with the Cogntie Core Data Model, then I created a custom data model through neat and I need to migrate the assets from the core data model to the custom extended data model, I tried creating a transformation but got the following error: failed with status 403: Properties [assetHierarchy_path, assetHierarchy_path_last_updated_time, assetHierarchy_root] are maintained by DMS and cannot be modified by end users. is there a way I can migrate/update the core data model assets to our extended data model?

Hi,Context:We currently organize our data in CDF using a per-country partitioning strategy, where each country has its own space.This approach was chosen primarily to restrict data access by country in a fine-grained manner.On top of that, we expose data models grouped by Business Object, such as Well Architecture, Cost Model, etc.Each Business Object model aggregates several data objects under a common business theme, which also allows us to control access by business domain in addition to country-based access.We are now planning to integrate a large amount of historical data, which will likely increase our model size by around 4x.These historical datasets are rarely queried, but we want to make sure their addition does not degrade performance for operational data — both in query latency and data ingestion throughput.We are evaluating two potential strategies: Keep everything in the same spaces, adding an indexed attribute (e.g., is_legacy = true) to distinguish legacy records. Crea

While I try to insert a Stream (using the http post as shown in the Getting Started guide), I am getting the next error:cognite.client.exceptions.CogniteAPIError: Project 2982735218914002 not enabled for Industrial Log Analytics | code: 400 | X-Request-ID: 66573df8-87ce-9b37-a837-180aee796a6c | cluster: westeurope-1 | project: slb-mmv-ops-dev This detail isn’t mentioned anywhere on the documentation. I cannot find how to change that. Can you enable ILA for https://delfi-dev.fusion.cognite.com/slb-mmv-ops-dev/?

I am stumbling through the Beta documentation. When I try to run the http post to create a Stream (as described here), I get this error:The feature can only be used with [alpha, beta] headersI found documentation that seemed relevant, but it seems the format of the header is this cdf-version: 20230101-beta https://api-docs.cognite.com/20230101/#section/API-versions/Beta-versionsHow do I know what date should I put into the header to gain access to the Streams beta feature?

Hi,I have been updating the beta documentation a bit since the last edition linked in the invitation. This applies to both the planned standard Cognite documentation, and the developer API documentation. All of the documents are available using a direct link to our document rendering services (linked below), and should be updated as we privately deploy new information.Note that all of these documents are works in progress with ongoing updates, so please forgive any typos, omissions, and other errors at this stage: Streams API documentation Records API documentation Capabilities for CDF Records Updated Data Modeling concepts page Concepts page for CDF Records (and Streams) Example high level use case (alarms) for CDF RecordsPlease do not share these documents.

Hi team,  do you have 2-3 articles on how to get started, like a simple quick start example as a starting point to be acquainted with the service ?Thanks !FYI ​@Marwen TALEB 

Is there a way to delete the relationship between an instance and a container? Let’s say I have a view Foo in one data model and another view FooExtension in another data model. FooExtension implements Foo.I have an instance of externalId “FOO”, space: “FOO-DAT” that have data stored in both views/containers.If it was no longer necessary to store data in container FooExtension is there a way to just delete this relationship in a way that `hasData(FooExtension)` from the DMS Query doesn’t find this instance anymore without affecting the data stored in Foo and the instance itself? Since instance count is limited, it’s becoming more common in our project modeling things having the mindset of shared instance when they mean the same thing across data models but with different formats to present the data on different Apps. But sometimes we don’t want to have the data in the app anymore and virtual deletion (boolean flags) is not always an option specially when the data volume is big. 

Hello, We are a heavy users of Grafana CDF datasource. Are you planning to support alerting anytime soon?  Thanks !

HiI am testing out event based triggers to see if its a good fit for a use case I am working on. When deploying the trigger the workflow is triggered many times until it has processed the whole input query, however as very many workflow executions are started at the same time, some of them will fail as the limit for running workflows has been reached.How is this handled by the trigger, will the failing workflow runs be retried? If not, is there a setting I can use to limit the amount of concurrent runs?I think this would be good both for distributing the workflow runs out and avoid the failures and also reduce the risk of failing function calls because of high load on the API.Thank you!Sebastian

Is there a way to configure a Hosted HiveMQ Extractor to extract the data into CogniteTimeSeries and not the traditional Time Series?

As illustrated in our documentation, the CDF Records feature uses a Data Modeling container as the schema definition for record data. I.e. you have to create a space and container first, before you start loading Records to a Stream. (Note: It is possible to use multiple containers together to define the schema for a single record. This may make sense in the context of, for instance, a Work Order record).However, at the moment when I write this, using the container based schema represents a somewhat confusing “limitation” when it comes to the size of a container vs the size of a record (number of properties). The way we have implemented this capability in CDF Records at the moment - using the DM containers - comes with a side-effect: The number of properties you can have for a single container is, as of right now, the same as it is in your Data Modeling service.The limits we're documenting in terms of properties for CDF Records are linked to the properties containing data within a singl

When running the following lines of code I get an error with Pydantic:from cognite.neat import NeatSessionneat = NeatSession(client)The error is as follows:---------------------------------------------------------------------------ImportError Traceback (most recent call last)Cell In[13], line 1----> 1 from cognite.neat import NeatSession 2 neat = NeatSession(client)File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cognite\neat\__init__.py:4 1 from cognite.neat.core._utils.auth import get_cognite_client 3 from ._version import __version__----> 4 from .session import NeatSession 6 __all__ = ["NeatSession", "__version__", "get_cognite_client"]File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cognite\neat\session\__init__.py:1----> 1 from ._base import NeatSession 3 __all__ = ["NeatSession"]File ~\AppData\Local\Programs\Python\Python311\Lib\site-packages\cognite\neat\session\_base.py:9 7 from

Hello,  We recently migrating the way of deployment of Streamlit App in our cdf toolkit project. Before we used to deploy our streamlit app by creating a file in a dedicated dataset. Now we use out of box way of deploying Streamlit Apps provided by cdf toolkit  (this was not available before).  When changing to this way, we notice that the app is only deployed using “--include streamlit” option.i.e: when not using the “--include streamlit” cdf toolkit the deploy summary is the following:    however, we notice it is a previous version that has been deployed!  when using “--include streamlit”: summary is the following And the App latest version is deployed as expected. Toolkit Version used is '0.6.20'. Are we missing something? Could you please take a look at this?  Thanks!      

When creating a Cognite Function you specify the dataset it belongs to, as an example lets say D2 LCI, then the files of the function is stored as files in that dataset. However, as D2 LCI is a dataset for file storage for engineering documents these function files cause noise, not much but still. If you were to list out all files for the dataset these “internal” CDF files would simply show as regular files and would have to be filtered out.Now, as we (AkerBP) are moving away from the asset-centric and over to data modelling this might not be an issue given how data models and spaces are. But in the future when datasets are a thing of the past, how will function and the like work, where will those files be stored. Storing the file used for compute with the data they compute might be a simple implementation, but not ideal as it does cause some noise. Note: this does not only apply to Cognite Functions, but to all CDF features where “internal” files are stored alongside the regular data.

Datasets have an “Access Control” page where you can see which groups have access. Is there a similar way to quickly identify which groups have access to Data Models / Spaces?If not, can this be implemented as a feature?

Hello,I’m looking to compute the standard deviation of a timeseries on the fly with synthetic timeseries.I expected to use this pseudocode formula : sqrt(avg(pow(TS{externalid}-avg(TS{externalid}),2))) with endpoint :client.time_series.data.synthetic.query(    expressions=expression,    start="2w-ago",    end="now")Unfortunately, avg expect at least 2 inputs, I try to switch to aggregate feature but I found it available only for timeseries, not synthetic timeseries.expression = '''sqrt( avg( pow( ts{ID} - ts{ID, aggregate="average", granularity="14d"}, 2 ) ))'''Do you have any tips or workaround to compute this value when “start” value changes ? Dont hesitate to explain I'm open to any opportunity to calculate this metric using another method.Thanks in advance,Pierre edit : I find this function in additionnal library : Rolling standard deviation of data points time delta — indsl 8.7.0 documentation but i’m looking for a answer without additionn

I need help getting attached error(Error: no gl) in chart page

Hello,  We are heavely relying on transformations to transform our data from Raw (staging) service to Data Models. We notice that it has very high latency compared to a “simple” Spark job. As per our discussion with our Solution Architect we understood that the bottleneck is the Raw service. Any plans to improve this in the future? Thank you!

Hello,I am working with the Cognite Data Modeling API to validate edges in a large data set. Our edges do not have a dedicated "edge view" defined in the model. When I attempt to use the instances.query() API to fetch edges filtered by their type, I encounter errors or unexpected behavior indicating that selected properties do not match the view schema.Specifically, filtering or selecting on properties such as "externalId" fails because the queried view does not explicitly contain those properties, as the edges exist only as instances without a separate view.Is there a recommended approach or best practice for querying and validating edges in scenarios where no dedicated edge view is defined? Should we rely exclusively on the instances.list() API with filters, or is there a way to construct valid queries for edges in the current model setup?Any guidance on how to effectively query and validate edges in such cases, especially for large-scale data, would be appreciated.Thank you!

Hi Cognite Team,I'm building a React dashboard using Cognite Data Fusion for my portfolio and would like to know if there are any demo projects available for working with CDF data.

We’re using the client.documents.previews.download_page_as_png_bytes() function in the Python SDK to retrieve png previews of P&IDs in order to display them in a webapp (Streamlit).For some P&IDs, the dimensions of the returned png are distorted. The image is in the correct orientation but squashed to fit within a portrait envelope. The affected P&IDs are displayed correctly in both CDF Data Management and when downloaded and opened in a pdf viewer.Has anyone had this issue before and, if so, did you find a way to handle it? 

Something is wrong with this overview. The fields CREATED/STARTED/FINISHED does not seem to correspond with what is shown in the graph, and the DURATION.Can someone explain what the STARTED field actually is, the graph and the timestamps don’t line up. Does it count the start of the transformation when it start writing or when it start reading (only one of these is sensible).The duration should be the time between the transformation was triggered and it was completed. From the perspective of the user the transformation has started when it triggers, how long it takes before it actually begins should be part of this duration. Measuring both intervals could be interesting, but the one that is currently there is not really useful for us. 

HiWe are testing out a new deletion transformation for a view in data modeling. The transformation looks something like this:select externalIdfrom cdf_data_models(...)where project in ("ProjectA","ProjectB")This transformation always times out and returns a “Graph query time out” error message. However, when changing the transformation to this the transformation just runs fine:select externalIdfrom cdf_data_models(...)where project = "ProjectA"unionselect externalIdfrom cdf_data_models(...)where project = "ProjectB"For context, the project column here is indexed and the view has around 10 properties and about 1.7 million rows. The transformations all query the same view.We are therefore wondering how the filtering in transformations handles filtering in data modeling and why these transformations are performing different. Learning more about this would benefit us greatly when working with transformations and data modeling in the future :)Thanks in advance!Sebastian

Hi When testing some new functionality in a Cognite Function I got this error message: “The size of the data field is too large.”The data input was from the output of another function ran previously in a workflow.I am looking through the documentation to find what the limits are, but can not find anything.For my use case I can decrease the input size and run more batches, but it would be good to know what the size limits are for input data into Cognite Functions. Also, are there any other size limits I should be aware of?Thank you!Sebastian