Skip to main content

Product Ideas Pipeline

Filter by Idea Status

Filter by Topic

1302 Ideas

Niranjan Madhukar KarvekarSeasoned ⭐️⭐️⭐️

How to use is_new() function with lastUpdatedTime from FDM type in transformationPlanned for development

Hi, When the data is in CDF RAW then we can use is_new function with lastUpdatedTime  in transformation and this works fine but how can we use that function for data coming from FDM model. Let’s say, from CDF RAW we are populating the data in FDM domain model and we are considering only incremental data with is_new function in transformations.This is the domain model (space: dm):type Entity @view(space: "dm", version: "1"){name:String} This is transformation query to populate above Entity type:select  distinct concat('wellbore:',WellUWI)  as externalId,  WellName as name, FROM  `db1`.`wellbores`where is_new(        "db1:wellbores",        lastUpdatedTime      )This is good and we can see only incremental data from CDF RAW in the domain model.  Now this is the solution model (sol_model):type WELLBORE{name:String }This is transformation query to populate above WELLBORE type ( I can directly map the name of WELLBORE of solution model to ENTITY of domain model but for this example I am ignoring it to understand is_new() function):select  entity_table.externalId as externalId,  entity_table.name as name FROM  cdf_nodes('dm','Entity','1') as entity_tablewhere is_new(        "dm:entity",        entity_table.lastUpdatedTime      )Here, I can’t use lastUpdatedTime because even though it is internal attribute of Entity type, it is not possible to get it in the result of cdf_nodes . The cdf_nodes doesn’t give us internal attributes. How can we make sure, the solution model transformations only consider the incremental data coming from domain model?  Am I missing anything here?   

Oussama ALLALI
Seasoned ⭐️⭐️
Oussama ALLALISeasoned ⭐️⭐️

Request for Asynchronous Functionality in Time Series Subscriptions APIImplemented

Dear Team,I am reaching out to provide feedback and suggest enhancements for the new Cognite subscription API, particularly concerning its use in continuous data streaming applications.Challenge with Time Series Subscriptions API:We are currently utilizing your Time series subscription API in a project that requires continuous data streaming. However, the API’s synchronous nature has presented us with significant challenges. The main issue is the inherent latency in data processing, which is exacerbated by the need to implement a sleep mechanism with a fixed delay. Determining the optimal sleep duration is challenging; too short a delay leads to frequent polling, while a longer delay results in data lag.Benefits of Asynchronous Streaming:Implementing asynchronous functionality in the API would greatly alleviate these issues. Asynchronous methods would enable more efficient and responsive data handling, reducing latency and providing a smoother data stream. This is particularly crucial in applications where timely data processing and analysis are vital.Suggestions for Improvement:We suggest the addition of asynchronous streaming capabilities to the API, such as non-blocking data fetches or real-time push mechanisms. This improvement would align with industry standards and significantly enhance the API's utility for various applications, especially those requiring real-time data processing.Best regards,Oussama