Skip to main content

Product Ideas Pipeline

Filter by Idea Status

Filter by Topic

1354 Ideas

HarshaSeasoned ⭐️⭐️

Cognite File extractor : Addition of Configuration to control the no. of files to be loadedClosed

I am writing to request a new feature for the Cognite File Extractor tool that would enhance its functionality and usability. Specifically, I propose the addition of a configuration option to control the number of files loaded by the extractor.As a user of the Cognite File Extractor, I have encountered situations where I need to manage large volumes of files during extraction processes. Currently, the tool loads all files present in the specified directory, which can lead to performance issues and inefficiencies when dealing with a high number of files. Therefore, implementing a feature that allows users to specify the maximum number of files to be loaded in a controlled manner would greatly improve the tool's flexibility and performance.The proposed configuration option would enable users to define a limit on the number of files extracted, through a numerical input . This would give users greater control over the extraction process, allowing them to tailor it to their specific needs and optimize performance based on their system resources and workflow requirements.By introducing this configuration control, users can effectively manage resource utilization and streamline their extraction workflows, leading to improved efficiency and productivity.  Also consider adding logging for this feature which will help to debug in case of issues. 

APSHANKAR Sagar
Seasoned ⭐️⭐️
APSHANKAR SagarSeasoned ⭐️⭐️

LLM proficient in using CDF APIs and Data ModellingGathering Interest

It would be nice if CDF offered an LLM which knows it’s eco system and can help build solutions. I tried to make one using CGPT-4 custom LLMs with the following prompt. However, it is prone to too much hallucination. Here is the prompt I used: This GPT understands fully the environment of CDF (Cognite Data Fusion). The GPT should be able to distinguish between all Cognite data structures (Time Series, Sequences, Events, Synthetic Time series, Assets, Data models etc. ) and how to work with them (CRUD, filter, aggregate..), understand the technologies available to use in CDF including their limitations on the platform (Streamlit, PowerBI etc.), understand Data Modelling in a CDF context (including CDF definitions for containers, spaces, direct relations, edges and views) and be well versed in the IO options to interact with CDF like the various toolkits and LATEST versions of SDKs (cognite-pygen, cognite-sdk, cognite-tk, cognite-logger, GraphQL etc.). The GPT should know everything that is published on [*.]cognite.com. (docs.cognite.com, learn.cognite.com, hub.cognite.com etc. and also cognite-sdk-python.readthedocs-hosted.com ). This includes all the case studies that can be found using Cognite.  The GPT knows everything there is to know in the public domain about Cognite Data Fusion. It also knows pandas, numpy and plotly. The GPT assumes Python11 is used for code and can also code in PowerBI, GraphQL. The GPT can leverage cognite-tk and GitHub, GitHub Actions to give advice and code suited for DevOps for Cognite projects including managing staging and production environments. I don’t know if I missed any public source but even an LLM trained only on the sources I mentioned will be very useful.