Solved

DB Extractor: Multiple query execution via single .ymal file

  • 6 January 2023
  • 7 replies
  • 87 views

Badge +3

How can I execute two or more different queries extracting the data from the source and inserting the data in the same CDF RAW table?
Currently only the last query available in the config.yaml file gets executed.
 

icon

Best answer by Pierre Pernot 6 January 2023, 08:37

View original

7 replies

Userlevel 5

Hi,
The DB extractor allows for running many queries, did you have a look at the logs ? Maybe there is something wrong with the first queries ? (setting your logger to DEBUG will probably give you more insights)

Badge +3

Hi @Pierre Pernot ,

I’m keeping the destination of both queries as some. Even setting the logger to DEBUG, I’m unable to get any other information.

 

Userlevel 5

In the logs, you should see if the config was loaded correctly or not. Could you check? If you edited your file, it could be that some formatting is wrong now, and the extractor is still using the old version of the config 

Userlevel 3

Hi @Prashant Chauhan, just wondering if you were able to sort out your issue?

Badge +3

Hi @Carin Meems ,

I’m still unable to resolve this.
Only the last-mentioned query is getting executed.

 

Userlevel 5

Hi,
In the configuration file you shared, I can’t see any dash ( “-” ). Can you please add a dash in front of each “name” argument ? 

In a YAML file, each list element must start with a dash. (https://docs.ansible.com/ansible/latest/reference_appendices/YAMLSyntax.html

queries:
- name: query_one
database: postgres
query: >
SELECT * FROM ITEM_PROPERTY WHERE PROPERTY_TYPE = "NAME"
destination:
database: "query_multiple"
table: "raw_one"
primary-key: "{ITEM_ID}-{PROPERTY_STRING}"
- name: query_two
database: postgres
query: >
SELECT * FROM ITEM_PROPERTY WHERE PROPERTY_TYPE = "PRODUCT"
destination:
database: "query_multiple"
table: "raw_one"
primary-key: "{ITEM_ID}-{PROPERTY_STRING}"

 

 

 

Please try and let us know.

Pierre

Badge +3

Hi @Pierre Pernot,

Thanks for your help.

Reply