New feature for SAP/DB extarctor

Thanks for logging this @shubha k v we’re looking into this feature and will followup as we consider it further.

Apologies for the slow response. Unfortunately, we had a bug which meant we did not get this feature idea reflected in our internal system for review until 2 days ago. 

Hi @shubha k v, and thanks for the request. 

This is probably my lack of understanding on full display, but I’m unclear as to whether you’re wanting us to add support/functionality to simplify reporting from the extracted data (if so, what kind of reporting/reports are you wanting to enable?), or if you’re asking us to add something equivalent to a “data last modified” timestamp to the base functionality for one or more of the SAP extractor variants listed on our extractors page?

Would be helpful to me if you would be able to describe the business problem you’re wanting to solve (that we’re not currently solving for you)?

Hi @shubha k v 

In the SAP Extractor configuration you have the option to add a list of fields to be used as keys in CDF raw. These fields will be concatenated during extraction, creating an unique raw key. Here is an extraction configuration example for MaintenanceNotification standard entity in S/4HANA:

  -
    source_name: mys4hana
    name: notifications
    destination:
      type: raw
      database: sap_extractor_v2
      table: notifications

    sap_service: API_MAINTNOTIFICATION
    sap_entity: MaintenanceNotification
    sap_key: [MaintenanceNotification, LastChangeDateTime]

Result in CDF RAW:
 

Let me know if this suggestion works for you. Otherwise, please let me know more details your needs so we can evaluate this further as a feature request.

Regards,

Rogerio Junior
Cognite Engineering

I want support/functionality to simplify reporting from the extracted data.

In my case need to add report_date field to primary keys even tough its not available

Rogerio Júnior wrote:

Hi @shubha k v 

In the SAP Extractor configuration you have the option to add a list of fields to be used as keys in CDF raw. These fields will be concatenated during extraction, creating an unique raw key. Here is an extraction configuration example for MaintenanceNotification standard entity in S/4HANA:

  -
    source_name: mys4hana
    name: notifications
    destination:
      type: raw
      database: sap_extractor_v2
      table: notifications

    sap_service: API_MAINTNOTIFICATION
    sap_entity: MaintenanceNotification
    sap_key: [MaintenanceNotification, LastChangeDateTime]

Result in CDF RAW:
 

Let me know if this suggestion works for you. Otherwise, please let me know more details your needs so we can evaluate this further as a feature request.

Regards,

Rogerio Junior
Cognite Engineering

If last change date field is not availble in our source system then how can we handle this

@shubha k v ,

If last change date field is not availble in our source system then how can we handle this

Since LastChangeDateTime is supposed to be included for standard SAP tables, have you tried verifying that ithe data doesn’t exist on the source system?

Hi Thomas,

Yes LastChangeDateTime  is not available as its SAP standard API. 

And what data we get daily they want to  append the data and upsert

After investigating, we've determined that we do not want our extractor(s) to add and append data as real data is being extracted. Closing as "Won't do"