Cognite File extractor : Addition of Configuration to control the no. of files to be loaded

I am writing to request a new feature for the Cognite File Extractor tool that would enhance its functionality and usability. Specifically, I propose the addition of a configuration option to control the number of files loaded by the extractor.

As a user of the Cognite File Extractor, I have encountered situations where I need to manage large volumes of files during extraction processes. Currently, the tool loads all files present in the specified directory, which can lead to performance issues and inefficiencies when dealing with a high number of files. Therefore, implementing a feature that allows users to specify the maximum number of files to be loaded in a controlled manner would greatly improve the tool's flexibility and performance.

The proposed configuration option would enable users to define a limit on the number of files extracted, through a numerical input . This would give users greater control over the extraction process, allowing them to tailor it to their specific needs and optimize performance based on their system resources and workflow requirements.

By introducing this configuration control, users can effectively manage resource utilization and streamline their extraction workflows, leading to improved efficiency and productivity. 

Also consider adding logging for this feature which will help to debug in case of issues.