wtimsl metadata update option in extractors yaml file

No, there’s no feature to do this at the moment

​@Sebastian Wolter shall I go ahead and convert this question to a product idea?

Hi

Thanks ​@Einar Omang for the information!.

​@Mithila Jayalath I think would be nice to have that feature just like the PI Extractor which gives more flexibility to data extraction and transformation.

Thank you!

​@Sebastian Wolter Thank you for the confirmation. I moved this to the product ideas section.

Thanks ​@Mithila Jayalath for moving this to product ideas.

​@Sebastian Wolter I’ve notified the Product Manager responsible for Extractors, and expect them to either reach out directly, or connect with you through this thread should they have any questions or any further follow-up.

Thanks everyone for the replies and follow-up on this!

Hi ​@Sebastian Wolter,

Right off, I can think of a couple of scenarios where the overwrite behavior - always update - could be suboptimal. So I’d like some feedback from you in terms of what you’re thinking for the following scenarios?

1. Write metadata initially, then stop writing it (one time/initial write)
2. Add metadata keys that did not previously exist in the target resource (along with the new value)
3. Update only values, and ignore new keys for the target resource
4. Only update metadata values that match a given configuration filter for the key(s)
5. Only update metadata values that have changed in the source as compared to the target resource
6. Always update existing keys and values (ignore checking for existence or value change)

Hi ​@Thomas Sjølshagen 

Yes basically our work scenario would be aligned to your scenario No. 1 and 3, so for say first time TS are created all info is saved as it comes, then after that (as new values come) only values are updated but no metadata

Why? Because we are trying to run transformations to update some data and metadata fields, but as new data comes they are overwritten every time new data arrives and the transformation become useless, and will need to be run only once all data is downloaded and extractor is stopped, but we need to update those filed in real time.

thsank you,