Skip to main content

Product Ideas Pipeline

Filter by Idea Status

Filter by Topic

1353 Ideas

Lucas Rosa AlvesSeasoned ⭐️⭐️

Storage for Intermediate Workflow tasksGathering Interest

SummaryWhen building workflows in Cognite Data Fusion to populate views in Data Models, there is often a need for intermediate, curated datasets between raw source data and the final target data model.Today, this intermediate data can be stored in RAW tables, but that requires customers to manage temporary tables, cleanup logic, naming conventions, and lifecycle handling manually. A native, workflow-managed temporary storage layer would make workflow development cleaner, reduce repetitive transformation logic, and simplify the overall implementation.  Business ContextWe are using CDF workflows to populate an Asset Hierarchy data model.The source data comes from multiple systems:SAP Functional locations Equipment AVEVA PI Tag metadata The target asset hierarchy data model contains the following views:Site Area Line Equipment System Subsystem TagThe source metadata arrives without the required treatment, standardization, or contextualization. Before writing to the final data model, the data needs to be cleaned, normalized, enriched, and structured according to the target hierarchy.Current ChallengeIn practice, the workflow needs several intermediate transformation steps before writing to the final views.For example, the workflow may need to transform SAP functional locations into a cleaned and standardized structure before deriving sites, areas, and lines.Example flow for Site: tb_functionalLocation  -> tb_functionalLocation_curated  -> tb_site_curated  -> Site view Example flow for Area: tb_functionalLocation  -> tb_functionalLocation_curated  -> tb_area_curated     uses tb_site_curated for contextualization  -> Area view Example flow for Line: tb_functionalLocation  -> tb_functionalLocation_curated  -> tb_line_curated     uses tb_area_curated for contextualization  -> Line view Example flow for Equipment: tb_equipment  -> tb_equipment_curated  -> tb_equipment_contextualized     uses tb_line_curated / tb_area_curated for hierarchy mapping  -> Equipment view Example flow for System and Subsystem: tb_functionalLocation + tb_equipment  -> curated functional location and equipment tables  -> tb_system_curated  -> System view tb_functionalLocation + tb_equipment  -> curated functional location and equipment tables  -> tb_subsystem_curated  -> Subsystem view Example flow for Tag: tb_tag  -> tb_tag_curated  -> tb_tag_contextualized     uses equipment/system/subsystem curated data  -> Tag view These intermediate curated tables are useful because they allow the workflow to:Reuse cleaning and standardization logic across multiple transformations. Avoid duplicating the same transformation logic in every step. Avoid using the final data model views as inputs to transformation logic. Keep the workflow logic easier to understand and maintain. Separate raw source data, intermediate workflow state, and final modeled data.However, this intermediate data can be transient. It is only needed while the workflow is running. After the workflow finishes successfully, the data can be deleted. There is also value to optionally allow users to view this intermediate datasets to analyze/debug the quality of the contextualization. Today, we can use RAW tables as intermediate storage, but this creates additional responsibilities for the customer:Creating and maintaining temporary RAW tables. Cleaning intermediate tables before or after each workflow run. Preventing stale intermediate data from being reused accidentally. Managing naming conventions for temporary workflow data. Adding cleanup steps to the workflow. Handling failed workflow runs where temporary data may be left behind. Writing additional code that is not part of the actual business transformation. Product IdeaIntroduce a native workflow-managed temporary storage capability in CDF.This could work as an internal temporary storage layer for workflows and transformations, where intermediate datasets can be written and read by different workflow steps, but their lifecycle is managed by the workflow execution itself.Ideally, this temporary storage would be:Scoped to a workflow or workflow run Usable by transformation steps Automatically cleaned up after successful execution, while still allowing end users to later review intermediate datasets for debugging. Temporarily retained for debugging and/or review Separated from RAW and from the final Data Modeling views Managed by CDF instead of customer-maintained cleanup logic Expected BenefitsThis capability would make workflow-based data modeling pipelines much cleaner and easier to maintain.The main benefits would be:Reduced amount of customer-managed code. Less duplication of transformation logic. Cleaner separation between raw data, temporary workflow state, and final modeled data. Reduced risk of stale intermediate data impacting future workflow runs. Easier debugging and monitoring of workflow execution. More straightforward workflow design for complex contextualization processes. Better support for multi-step data preparation before writing to Data Modeling views.

Mario NobleCommitted ⭐️⭐️

Atlas AI feedback and suggestionsImplemented

Atlas AI – agent building and chatbot notesThe GoodAble to switch between list and tile view as well as Search. Having a Description is helpful. Nice that there are sample questions. Loading feedback. Ability to stop generation. Legal disclaimer near prompt area. Nice that it shows reasoning steps. Like the suggestions with Show More Bot background color is fine. Reasonable area to write a prompt. Great to be able to view the details using the info icon of what each agent is good at before switching to it.Could be improvedPriority 1Starting a new chat is “risky” because there is no history, copy or chat download/export functionality. Part of the point is to aggregate information for analysis later, sharing or collaboration Chat history/multiple threads Need a way to set expectations up front so that when a new chat is started, we can orient users as to what can and cannot be done. Description helps but gets cut off. Sample questions as defaults are not injected but auto triggered. This can be an issue since the prompt is most likely not exactly what they want. Need some sort of prompt library so that users will be presented with prompts that have been vetted, and we are sure will produce relevant answers and get them quickly started. So not just examples but starter prompts. When it suggests follow ups, it would be great if these may be clicked and injected into the prompt area (but not auto submitted). Priority 2I would expect the naming for Atlas Agents to be in Industrial Tools and Agent Library to be in the Atlas AI. Ability to choose to share threads Edit icon should maybe use a plus. I know Copilot uses that but it’s still odd. Not sure what to do about location action and listing. What does this mean or how does it affect bot? Needs link to terms or docs. No file attachment. Users will probably want need this to override on a case by case basis. Publish / Unpublish flow is a bit awkward when prompt engineering between multiple people. Prompt area should be fixed to a certain height (perhaps one half the viewport) and scroll or be resizable. The use of a full takeover modal with a close button is odd when starting a chat. Switching to a new chat when moving between agents without a chat history is somewhat dangerous since users will lose all their work. It’s good that we can switch between agents while in a thread. However, I would expect that when I am in the agent library, clicking on another agent starts a new thread with that agent. Returning to the original agent would show my previous conversation with the agent. Cannot generate tables easily. Although I did manage to do so one time. I easily miss My Agents and Published tab. This causes a starter problem. Would like to add multiple tools at a time and not need to add them one by one.Possible bugsSeemed to lose past chats bubbles after a certain amount of conversation. Reloading shows them. Need to verify. Ran into an issue when switching between an agent within a thread created a new chat.

Anders Brakestad
Seasoned ⭐️⭐️⭐️
Anders BrakestadSeasoned ⭐️⭐️⭐️

Atlas Skills are very hard to use in a devops settingGathering Interest

Hi!The Q2 Product Release describes the new Atlas Skills feature as “reusable, lifecycle-managed assets”. In my experience the current designmakes skills very hard to reuse makes it impossible to manage their lifecycleReusing skillsTo reuse skills means to me that I can add skills to my agents that have been authored by someone else for another use case. The agent builder in CDF lets me add new skills in two different ways:Search among the skills already added to the agent Upload a skill markdown fileSo the only way to reuse a skill is to download a skill from another agent and re-upload it to my agent? There is no lifecycle management here. Skill versions will become stale very quickly. You have no idea whether your skill is up to date or not.There is no repository of skills where I can search for something relevant to me. I have no idea what skills have already been created. There is no way to inspect a skill once it has been added to my agent (I need to download it locally). Is there any versioning of skills? No devopsThere is no real devops integration with the current design. I am using a modular setup for defining my agents:recipes/agents/my-agent/|-- config.yaml|-- instructions.md|-- skills| |-- skill-1| | `-- SKILL.md| `-- skill-2| `-- SKILL.md`-- tools.yamlThe point is that skills are defined in their own files. There is nothing dramatic about this modular setup, and you might say it adheres to normal practices. But, how do I get these skills added to my agent? As far as I know, I cannot reference these files to the toolkit and expect them to end up with my agent. This is very different from how tools are added to the agents. Here is my painful workaround:Deploy agent to CDF with toolkit without the skills Duplicate the agent so I get a personal copy Upload my skill files to this new copy Dump new copy using cdf dump agents Copy the skills section with external ids into my original agent specification Deploy agent to CDF with external ids to the skills produced in step 3.We need the features to support standard workflows. As a user I expect production grade releases to offer production grade solutions. DocumentationWhere is the skills feature documented? I am unable to find it. The Q2 Product Release refers to this page, but nowhere under Atlas AI are the skills documented.

Anders Brakestad
Seasoned ⭐️⭐️⭐️
Anders BrakestadSeasoned ⭐️⭐️⭐️

Support modular, file-based agent definitions in the Toolkit (skills + Python tools)Gathering Interest

SummaryThe Cognite Toolkit forces an agent to be defined as a single monolithic YAML file — config (name, id, description, LLM), instructions, the skills list (external IDs to pre-existing skills), and tools (including inline Python source) all in one file. We need a modular, file-structure-based agent definition that the toolkit parses at deploy time, so skills and Python tools can live as standalone, testable, version-controlled source files that are created and attached to the agent automatically on deploy. Pain pointsThe monolithic YAML blocks normal DevOps practice (isolation, unit tests, review, reuse). Concretely:Skills cannot be deployed from source. A skill should be an isolated directory containing a SKILL.md (possibly other supporting files as well, as per the emerging standard). But the toolkit references a skill only by external ID — the skill must already exist in CDF. There is no way to point the toolkit at a SKILL.md and have it create and attach the skill on the fly during deploy. Python tool code is trapped inline. Executable Python tools embed their source directly in the agent YAML, so the code cannot be unit-tested or run standalone during development. Functions already support a modular layout — a directory with utilities/supporting Python files that gets packaged on deploy — and Python tools should work the same way.My opinion is that you need to keep these things in mind whenever you introduce new features. We as the users need to be enabled to use workflows that follow best practices. Devops should not be an afterthought, but be baked in from the start.

Gunnar AndreasSeasoned ⭐️⭐️

Configuration of the default View in Search, and the order they appear in SearchGathering Interest

With the data modelling framework, most models will end up with a significant number of Views. Today we are not able to control the order the Views are presented in a good way. By reverse engineering it seems like the order of the Views is based on The order of the (first mentioned) CDM extension in the model definition. Alle non-CDM Views are listed at the end, irrespectively whether it is listed before CDM or not  What is considered a CDM extension by the UI is not based on the implements, but whether it reference at least one property from one of the CDM containers Inside each Category (eg, a specific CDM extension, or the non-CDM Views), the order in the model definition dictates the orderIf I choose alphabetic sorting it seems to only affect the non-CDM-Views, and not the order inside each CDM category. It do however put all the CDM categories first or last It is very inconvenient to use the order of appearance in the model definition as the way of sorting (we have to sort both the CDM extension, and then inside each category) since any change in the order will require pushing a new model definition to CDF.We need the ability to define the order it appears in the UI, independent of how it is listed in the model definition, and we need to be able to create groupings that do not follow the CDM extensions. Eg, we have maintenance information that are not a CogniteActivity extension, that we still want to group next to the other CogniteActivity extended maintenance information.The CDM based grouping is more relevant from a model developer point of view, and not that much from an end user point of view.We also need the ability to define what is the default View. Right now it seems to be the first View in the model definition that is a CogniteAsset extension.When changing location filer, it always reset to the (non-configurable) default, which we see create a lot of initial confusion with the users. Since the CDM Views are not explicitly described in our model, we do not see the CDM based grouping. 

Kubota MamiSeasoned ⭐️⭐️

【Canvas】Feature Request: Improvements for Canvas Usability and Operational EfficiencyGathering Interest

We create a Canvas like the attached example once a month to monitor long-term trends of key operational parameters.Previously, this work was done using Excel, but we have recently migrated to Charts and Canvas.As we move toward full-scale operational use, we would appreciate your support in resolving the following two challenges.These improvements are critical for reducing operational workload and enabling sustainable use of Canvas for monthly monitoring.Request ①We previously requested a permission transfer feature for Charts. However, it is currently difficult to locate individual Charts from within Canvas.Cognite HubTherefore, we would like to request the following enhancement:Ability to select Charts directly within Canvas and transfer their ownership/permissions in bulkWithout this capability, when the responsible person changes, users must first locate all relevant Charts individually and then manually duplicate them one by one. This process is time-consuming and creates a significant operational burden. Request ②When downloading the entire Canvas as it is, each Chart becomes too small to view clearly, making the feature practically unusable.Each month, we include “Special Notes” (as shown in the attached example) and review selected Charts together with those notes.Therefore, we would like to request the following functionality:Ability to select specific Charts on Canvas and:Extract only the selected Charts Automatically format them into an A4-sized layout Export both the visual output and the underlying data (CSV)This feature would enable us to generate clear and usable reports for monthly reviews. Resolving these issues is essential for enabling effective operational use of Canvas.We would greatly appreciate your consideration and support.

Prashant ChauhanPractitioner ⭐️⭐️⭐️

Dynamic Space Assignment in DB Extractor QueriesGathering Interest

Problem StatementThe Cognite DB Extractor currently requires space to be a static, query-level configuration parameter, forcing users to create multiple identical extraction queries when distributing data across spaces based on hierarchical entity relationships.When working with hierarchical data structures, each distinct parent entity requires a separate extraction query, even if the source data and transformations are identical.Asset hierarchy:L1: Assets  L2: Area L3: Field (defines CDF space) L4: Installation L5: WellCurrently, if there are 10 distinct fields across multiple assets, the DB Extractor requires 10 separate extraction queries to distribute timeseries data across 10 spaces - one query per field, even though they all read from the same source table with identical transformations. With multiple assets, areas, and fields, this results in hundreds of extraction queries for a single source table. This creates:Configuration Bloat: Hundreds of nearly identical YAML configurations to maintain Filter Duplication: Adding or modifying a filter condition requires updating hundreds of queries Scheduling Overhead: The same database table is scanned multiple times per ingestion cycle instead of once Resource Inefficiency: Hundreds of independent jobs running on the same data multiplies query load Error Management: One failed field extraction leaves only that space with stale data; manual recovery required per space Maintenance Burden: Updating logic across all queries is error-prone and time-consuming Scalability: As fields are added or wells multiply, query count grows exponentiallyProposed Solution:Allow space to be derived dynamically from query result columns using template syntax, similar to how external_id is generated.