Community Discussions

Hi,I am working on one of the project where I am using MQTT extractor to inject data into CDF. Kindly advise best optimized stepwise solution to integrate and transform data into CDF.Note that it is Sparkplug B-formatted MQTT data.Kindly advise.Regards,Amol S Pawar

Hi Team,I'm excited to be registered for the Data Engineer Basics course, specifically the “Learn to Use the Cognite Python SDK” section!While working through the “List, Search, Retrieve” portion, I came across an issue. The documentation states that the general pattern for search operations is `client.<cdf_resource_type>.search()`, where `<cdf_resource_type>` could be `data_sets`, `asset`, `time_series`, `events`, `files`, `labels`, and so on.Following this, I tried the code:```pythonc.data_sets.search()```However, it resulted in the following error:```AttributeError: 'DataSetsAPI' object has no attribute 'search'```I double-checked the Python SDK documentation but couldn't locate a `search()` function for `data_sets`. Could you please let me know if I missed something? And if the `search` function for `data_sets` is indeed unavailable, perhaps it could be noted in the documentation to prevent futu

Hi!I have a question regarding how quickly the metadata for a FunctionCall is updated after a successful call. The reason I ask is the following case:I have deployed a set of Cognite Functions that run on a schedule every minute. In production this will be every 10 minutes, but for testing and debugging I want to speed it up to see how it works.One of the functions perform this job:1. Determine the time range for the data fetch based on response value of previous call (if first call we use pd.Timestamp.now())2. Fetch Events in time range and process the data3. Write processed events to a Sequence resource in CDF4. Return a dict {"status": "success", "start_time": start_time}The start_time of call N becomes the end_time for call N+1. The function is a “historic backfiller” that traverses back in time to a pre-defined date and fetches Events along the way.The issue:I have run into the scenario that in

Is there currently, or is it planned on the Cognite roadmap, a Job UI for real-time tracking of job statuses, including runtime, failure reasons, and retry counts? Additionally, will it provide a step-by-step view of each job’s progress to help easily identify and troubleshoot issues?

Hi Developers! :)I have a use case where I need to retrieve a large number of Time Series data points from CDF. We are a final DataFrame with a shape of approximately (1000, 400000).  So a total of 4 billion data points + the index. I also want the data retrieve to be fast…. :)  So instead of retrieving everything in a single call to `client.time_series.data.retrieve_dataframe`, I thought to use multithreading to start the retrieves in an approximately parallel fashion, and then concatenate the resulting dataframaes at huge runtime gains! However, I observe that the results from multithreading do not equal the results from the single call. :( Specifically:I split the data retrieval on Time Series IDs, and concatenate along axis=1. I observe that data from the “first two” threads have a lot of missing data, while data from the last two threads are 100% correct. I have also tried to split the DatetimeIndex and multithread along this dimension,

I would like to know if Cognite visualization has the same features as the one found in Power BI and Tableau.The reason is simply because we need to have Decision Support and Visualization module as part of our overall setup.

Hello,I’m trying to pull energy usage data from a Rockwell power monitor that supports OPC-DA but not OPC-UA. I configured the OPC-DA server (rslinx classic in this case) and I’m able to see the live data using an OPC-DA client (Kepserver). Next I try running the extractor, and it sees the project structure and creates assets, but never creates any time series data. I’m seeing very little in the logs. I did notice that “contious: False” shows up, but I don’t see any way to change that via configuration. I will get a keep alive message as you can see in the second image, but no data. Any suggestions would be greatly appreciated! Thank you.  

Hello everyone,I have a problem viewing 3D. It tells me that I don't have any scenes created, but I do have some. I understand that the error message is because I have to configure the location in the scenes. How do I do that? I don't see the option to add the location.  

Hi Cognite Team,I'm curious about the specific rules you follow for versioning views, particularly with the versioning format like "@view(version: 'v4')". I’ve noticed that the last number is the only one that changes, and I wanted to understand the rationale behind this.In my experience, I typically use the X.Y.Z format, where:Major versions indicate significant changes or potential backward-incompatible updates. Minor versions introduce new features or enhancements. Patch versions address bug fixes or minor improvements.While I understand that tools like DVC are more suited for tracking actual data changes, we are focusing on versioning the views and models. Is this approach aligned with your guidelines?If there aren’t any formal rules, that’s perfectly fine; we’re just trying to establish our own conventions and best practices. Versioning is an important aspect for us, and any insights you could share would be greatl

Hi everyone,I’m facing a challenge with our current setup and would appreciate some guidance from the community.We have an OPC-UA server running on a Siemens IPC Linux machine, which serves the variable values of a Siemens PLC. This OPC-UA server is queried by an OPC-UA client inside a Docker package (Cognite OPCUA Extractor) that sends the data to Cognite Data Fusion (CDF). The data points are then stored in their respective time series within CDF.The issue is that, during periods when the physical quantity doesn’t change (e.g., motor current at zero during weekends), no new data points are sent to CDF. As a result, when the motor starts again, and the current shoots up quickly (for example, up to 2 Amps in 40 ms), this rapid change is not properly reflected in the time series. Because we only scan every 100 ms with the PLC, and since no data was logged during the weekend, it appears in CDF as though the current gradually increased from 0 to 2 Amps over the ent

In a hypothetical scenario where your company has more than 100 sites across different countries, each containing multiple units, it's important to organize data in a way that ensures scalability, flexibility, and ease of maintenance.We have our own perspective on the matter, but we would like to hear from other specialists, especially those experienced in data modeling approaches, like yourself. Example 01: Managing Employee Information Inside CogniteQuestion: Should we separate SPACEs for each site? Example Structure in Cognite Data Fusion:You can structure the SPACEs for your company’s sites and units as follows: Top Level: Country Create a SPACE for each country to organize data regionally.Example: US, BR, IN (for the United States, Brazil, and India). Mid Level: Site For each

Hi.I would like to experiment with using more of the advanced functionality in data modeling. For instance, everything related to importing views and mapping properties between data models. However, I feel like the documentation (https://docs.cognite.com/cdf/dm/dm_graphql/dm_data_modeling_language#specification-of-directives) is a bit lacking of examples of how to use the functionality. Are there any end-to-end (both basic and advanced) examples or other training material of how to reference data in other data models using all the different directives specified. For example examples of best practices when referencing between a source model and a domain model, and when referencing between a domain model and a solution model.Thank you!Sebastian

Hi everyone, I am wondering if someone can help explain the difference between Node/Edge and View/Connection Properties. I am extremely confused on these concpts as I read through the CDF data modelling tutorial.My current perspective is that Node/Edge and the knowledge graph defined by Nodes and Edges is the most “vanilla” version. Where as a Data Model made up of Views and Connection Properties is a similar construct but based on a specific perspective. The analogy between a View and Node in a relational DB sense will be a table and a view.Is my understanding correct? If so, I am wondering what is the actual need of Node/Edge given there is already Containers that perform the data storage like a SQL table would do? It seems to me that View, Connection Properties and Data Models are sufficient enough to produce any type of sementic model?

When I am creating calculations within Charts I often have to click an object twice before it is selected.  For example, adding a Function.  When I select Operators the list resets and I have to click Operators again.  This time it provides the new window.  I’ll scroll down and select Round (doesn’t matter what I select).  The list resets back to the top and I scroll back down and select Round again.  This happens every time independent of browser (Chrome or Edge).  Is this common for others or a unique issue for me?  

Hello, We are trying to delete a unit on a container attribute using CDF toolkit. CDF toolkit identifies that there is a container change. However, this change is not reflected on the container and view schema after the deployment. No error raised.Note: Adding and modifying container attribute unit works well.Any ideas ? Thanks,

Is it possible to get Cognite Data Fusion APIs from Minitab application? If does, how does it work the connection?

BackgorundWe have an ongoing project where we need to deploy Cognite Functions and other resources (TimeSeries and Sequences) regularly, both during development/debugging and when publishing new versions of our product.The way we have solved this today is this:We maintain a “deployment template” YAML file that defines all the “stuff” that needs to be deleted/created during a redeployment. This includes Cognite Functions, TimeSeries, and Sequences. Each entry in the YAML contains all necessary data for creating the relevant resource. We have written some Python classes that perform resource specific deployment (CogniteFunctionDeployer, TimeSeriesDeployer, SequenceDeployer). These behave quite similarly, with methods that backup data, delete, and recreate the resource. These classes are instantiated in our “deploy.py” script, which just itereates over all the resources defined in the template (step 1), and calls the “.deploy” method on eac

Hi,I was testing annotating P&ID documents towards nodes from a data model. It worked and the annotations are available through the API, but is not showing in the preview of the file. When the annotations are towards assets or files (from the asset-centric model) then it shows as expected, but when towards a node it isn’t. I am not sure if this is a bug or a future feature yet to be implemented.To be clear, I am annotating a File (not a CogniteFile from core data model). And the annotation is created and connected to the file and points to specific instances in a data model, but I see no way to visualize this is in Fusion. Regards,Markus PettersenAker BP - Data Delivery

Hi everyone!  🔔 Just a quick reminder that our comprehensive, self-paced, and on-demand courses are available at Cognite Academy. Whether you're looking to enhance your skills or explore new areas, we've got something for you! Upon completing a course, you’ll earn a certificate that will be added to your community profile and can also be added to your LinkedIn Certifications, helping you showcase your achievements to a wider network.Looking for more hands-on learning? We also offer 5-day intensive in-person trainings for those wanting a deeper dive.Unlock your next learning journey today!🚀

Hello everyone,I am using cognite client and wondering if I can write simple graphQL queries and run it through. Example. Select all assests.But it seems, it only provides option using one of the data models. Newbie to CDF. 

Hii everyone, I have created a data model and in that model I have one parent view and then a nested view. So I made cursorable true for parent and cursorable true for nested view in the indexes field(@container) so only limited rows of nested columns are coming into dataset( used a plugin to fetch dataset from cdf using dataiku) but if I do cursorable false for the nested view then all the rows are coming for it. Just for FYI in our code of plugin we have set limit of 1000 rows so when cursorable false then all the rows are coming and when cursorable true for nested view then only 1000 rows are coming. And when parent view’s cursorable is false it says it does not exist.Is this correct behaviour of the cursorable feature? if yes can you please explain why?

Hey Everyone!Curious about how to boost your rank and unlock some amazing new abilities in the community? 🚀 If you haven’t checked it out yet, we’ve just shared an exciting update about the perks awaiting you when you reach Level 4 (Practitioner). From marking correct answers to unlimited post editing, these benefits are designed to empower you as a community leader.👉 Check out the full details here!👈Ready to take your engagement to the next level? Head over to the original post for all the details and start leveling up today!

Hello !  I am new to this community but would like to ask the following:   Has anybody any experience extracting a list of DCS Process Alerts to Operators or Operator Overrides on an Hourly or Daily basis into CDF ?   This data resides in the OT and is sometimes extracted and made available in reports in the IT domain, e.g., by Yokogawa.   As I see it, this is an important indicator for stable and well managed production operations, and also a good predictor of upcoming threats to production.

The buttons have disappeared in the search after one of the releases. It is still usable, but you have to guess the functionalities. 

It was requested by @ibrahim.alsyed  from Celanese that we increase the limit of the raw datapoints for the endpoint /timeseries/data/list (Retrieve data points) to 1 million. Currently, the non-aggregated data points returned is limited to 100000.For the drill-down views with more than 7 days, the server-to-client data transfer maximum size is reached due to the number of datapoints (some timeseries have more than 1 datapoint per minute). The solution they have implemented is that for ranges higher than 7 days, they are displaying an interpolated trend using the maximum absolute values for 30-minute aggregation. Once the user selects a smaller date range, they are unable to display all the values. Hence a increase in the limitation was requested.