Product User Community

Hello Community,cdf stores the data in cloud, could  anyone know what is the database / database structure being used,and any reason for using Spark SQL for transformations.

Can someone revert with proper procedure for using any of the cognite data fusion API using postman 

Hi Team, From the cdf-fundamentals training, I have mapped required details but data is displaying as a linear format not trends.   

Hi there,Are there any plans to make the PI Extractor open source, following the same approach as opcua-extractor-net?

Hi! We have made the following updates available for Workflows.Optional TasksDefining non-critical tasks in a workflow as optional, meaning that in the event of a task failure, the workflow itself should continue, has been a much requested feature. You can now set the policy for how to handle failures and timeouts for each task in a workflow. By default, the workflow will fail if a task fails (after retries) or if the task times out. Alternatively, you can set the parameter to skip the task in the event of a failure or timeout (after retries). This means that the workflow execution will continue even when the task does not complete successfully. The feature is documented in the API specififcation here (navigate items > workflowDefinition > tasks > onFailure). Concurrency Policies for TransformationsIf a job is already running for a given Transformation in CDF, a new job cannot be started concurrently (fails with error message “A job already runs for this transform”). To better

Hi Community Members!We’ve recently published a cloud-based Virtual Flow Metering (VFM) system tailored for a late-life well with high water cuts. This solution employs a hybrid physics-data approach, delivering precise and near-real-time estimates of water production per well in gas fields.Link to the full article: https://www.sciencedirect.com/science/article/pii/S277250812300042X Enhancing Production OptimizationIn the oil and gas production domain, understanding the fluid composition and production rates of multiphase operations is crucial for effective planning and optimization. With wells yielding a diverse mix of oil, gas, and water, often with varying compositions, ensuring accurate measurements is essential for maximizing production efficiency. Ratios change in time, and this gets more and more important as the water cut grows in the late part of a water flooded well life. This article explores how Virtual Flow Meters (VFMs) offer a cost-effective alternative to traditional me

Hello everyone,I'm relatively new to using Grafana, and after reading this post (Cognite Hub) on Cognite Hub, I've come to understand that Alerts in Grafana may not be compatible with the CDF connector. Please feel free to correct me if I'm mistaken.Considering this, I'm wondering if anyone has a good setup for monitoring the data coming from the CDF connector. I have around 900 time series, and listing all of them is not very efficient. Additionally, I'm having trouble figuring out how to write an SQL query to count the time series that have a timestamp older than, let's say, 3 hours. this could be a way, to count data not coming in. Would anyone be willing to share their setup or offer any tips?Thank you in advanced :) 

In September, we embarked on an incredible journey with our partners at CNTXT Academy, bringing Cognite Data Fusion (CDF) training to the heart of Saudi Arabia. We were thrilled to host not one, but two engaging Cognite Data Fusion Fundamentals sessions right at the epicenter of innovation - Saudi Aramco's headquarters in Dhahran. Our participants came from diverse backgrounds, each with their own unique experiences with Cognite Data Fusion. So, we tailored the training to ensure it resonated with everyone, addressing their everyday work challenges that Cognite Data Fusion can brilliantly solve.The first part of the training focused on contextualization, mastering Cognite Charts, and seamlessly connecting CDF with Grafana to tackle a real-world use case. The participants got hands-on experience, putting their newfound knowledge to the test right in CDF. In the second part, our exceptional instructors introduced participants to the Cognite Python SDK, taking their skills to the next lev

SOLVED:The Cognite client was set up wrong, hence the Cognite API error when uploading data to CDF. See docs posted by Peter for setting up config.yaml for the extractor. Specifically these were the changes:``` COGNITE_BASE_URL=https://westeurope-1.cognitedata.com (not Fusion URL)COGNITE_PROJECT= {some project name}COGNITE_TOKEN_URL= https://login.windows.net/{some_tenant}/oauth2/v2.0/token(Different from OpenID connect token URL)```_______________________________________________Hello curious and helpful members of Cognite Hub,I’m building my first REST extractor to GET a response (sensor data) from a camera_sensor API, where I then aim to upload the data to corresponding timeseries. I have multiple cameras with each its URL path, so it seems @extractor.get_multiple(paths) gives me the desired behaviour of essentially looping @extractor.get through the URLs. However, this error arises when my extractor tries to upload datapoints to the timeseries in CDF.^CTraceback (most recent call la

Hi, I want to transform JSON object to Data model column. JSON is populated from extractor to raw table, so its type is by default Object and its structure will be nested.Is there any built-in/ Custom function using which I can parse this json  and convert the specific value into column. I have tried to_josn () which will convert the json structure to string and then with the help of get_json_object() I can get expected value by passing json path to this function. But problem here is it will always return value as String.  

Hi, I have done the Grafana setup from azure managed grafana from azure portal. But don’t have permission to install cognite data fusion in grafana from azure. Please let me know how to install plugin and what permission needed.   

Hi,I am running the DB Extractor, where my Source is an odbc connection(SQL DB) and destination is Timeseries. But I am getting the error: “ [ERROR   ] QueryExecutor_0 - Unexpected error in UB-Training-Timeseries-objects: 'NoneType' object has no attribute 'timestamp'”Attaching the screenshot of my data in SQL, error Log and config file.I am using DB extraction v3.0.0.Please reach out if you need more info.

Hi! We have made the following updates available for Workflows.Support for Workflows in the Cognite Python SDKThe Cognite Python SDK now has native support for Workflows. Documentation can be found here. We have also updated the example notebook, which you can find attached as a file below (Note: Download the file and remove the .txt postfix to open it as a Jupyter Notebook.)New Access Management CapabilitiesYou can now configure access to Workflows directly in Access Management in CDF. Navigate to “Manage” > “Manage access” and for the relevant Groups add capabilities to read/write Workflows as needed (see screenshot below). This replaces the experimental ACLs which were required initially to interact with the service. The capabilities can also be added to a group using the SDK directly, e.g.:from cognite.client.data_classes import Groupcapabilities = [{"workflowOrchestrationAcl": {"actions": ["READ", "WRITE"], "scope": {"all": { }}}}]group = Group(name="Workflow Orchestration", ca

Learn FDM GraphQL Queries . Aggregate query not supporting sort functionality. How to achieve sort functionality in aggregate  

How to fetch documents from multiple folder locations using single config.yml file by Cognite Documentum Extractor? how to write multiple DQL queries in same config.yml? Which configuration parameters to use?I am working on Cognite Documentum Extractor and using DFC Java SDK mode for connectivity.

Hi! I am trying to update “Planned End Date” for a specified work order in staging.I have prepared this table to join to my work order raw table:This is my sql and the error message I got:Is there a way I could update “Planned End Date”? Thanks in advance. :)

@Jørgen LundWe have a use case for Workflow Orchestration feature in a project – an example being we would like to run some workflows (Cognite functions) after successful runs of certain workflows as they have a dependency between them.There are some conversion and allocation factors that vary periodically and will be updated to SharePoint via end users. Based on the dates for these numbers, we will utilize them for further data munging and calculations in other Cognite functions, given that the previous function run was successful.Can we use workflows for this? If yes, can we enable this in our project?

Hi Team, IS there any unit testing examples for cogex based extractors to test integration with rest api

Hi! I have a question about the data modelling in Cognite Data Fusion.We record data from thousands of sensors from a factory. Today we record them as time series in CDF and organized in the asset hierarchy by manually constructing the relationships.I heared the data modelling toolkit is meant to replace asset hierarchies approach in future. Could you please elaborate a bit on how Cognite would suggest to approach our use case, if we started from scratch today.Thanks!

I have a set of timeseries objects t1,t2,t3,t4,t5…..etc. Now I would like to create a derived timeseries objects based on the existing timeseries objects. For instance  T1 = (t1/1000) *141.5T2 = (t2/100 - t3)T3 = (t3/100 - 14)T4 = t4 * t2/100T5 = t5/100How do I perform these steps using cognite SDK. Is there a way to perform these operations?Please advise. Retrieving dataframe from data-points and then create new timeseries objects (T1,T2….) will be challenging and hence wanted if there is an optimal way to handle this in a better way. Is this possible using synthetic time-series?IS there some examples and code snippets that can be shared so that I can grasp them better?  

I have had this problem before but it fixed it self by just waiting. But is the connection to CDF down when you get this error in PBI? Details: "Cognite: Request failed: The remote server returned an error: (503) Server Unavailable. (Service Unavailable)"

We are happy to share our newest enhancement to the search experience within Cognite Data Fusion, furthering our ambitions to deliver easy access to complex industrial data. This feature provides user-friendly search and enhanced context for industrial users, such as engineers, technical experts, and operational teams. You can narrow your search to a specific site - like an individual offshore asset or a processing facility - to make the search results more relevant The search results are grouped and presented in an industrial context and can be viewed through a list view and a 3D view with a wide set of filter capabilities across all given data types You can open data in Cognite Charts and Cognite Industry Canvas You can easily change between layered models to see CAD models, 360 images, and point cloud modelsWe are very happy to get feedback, and would love to hear how you experience this new feature. Best, Sofie, Product Manager 

Found a bug? Have a question about how something works? We want to hear about it!You can choose to either create a dedicated post (topic) in the Charts group by clicking the Create topic button OR simply post a reply below in this thread. Remember to include a screenshot or video to help the product team best understand what exactly you’re talking about or referring to.