Is there any way to read CSV files in an incremental manner from a drop folder following a file name pattern and ingest the data into specific RAW tables.
Eg. I have file names which have timeseries data from daily report with columns like asset id, timestamp, data point. With file name pattern as “timeseries_<unique number/date>.csv”. Users are pasting these files in a public sharepoint location in an incremental manner.