🚀 Impact 2024: The industrial data and AI conference for and by users | Nominate speakers now for a...
Hi @Kanchana Premachandra I have referred the provided files and now the extractor is working smoothly.I have updated support ticket. Thanks,Amit
Hi @Kanchana Premachandra Yes, I am using the configuration similar to above config.for reference:extractor:# (Optional) Number of rows to fetch from sources before uploading upload-queue-size: 10000# (Optional) Where to store extraction states (progress) between runs.# Required for incremental load to work. state-store:# Uncomment to use a local json file for state storage #local: #path: C:\Users\ARane3\Desktop\odf\ingestion\cognite-db-extractor\state.json# (Optional) Save interval (in seconds) for intermediate file saves.# A final save will also be made on extractor shutdown. #save-interval: 30# Uncomment to use a RAW table for state storage raw:# RAW database and table to use database: "state" table: "state:table"# (Optional) Upload interval (pin seconds) for intermediate uploads.# A final upload will also be made on extractor shutdown. upload-interval: 300 #upload-queue-size: 2databases: - name: CDF_SOURCE_DATA connection-string: "DSN=dsn;Uid=uid;Pwd=p
Hi @Sudil Abeyagunasekera The literals values may change based on condition that's why we wanted to get that result on runtime from the mapping.
Hi @Sudil Abeyagunasekera I have already tried the first 3 suggestion but still getting the 0 rows.for more reference I am using this query:SELECT concat(TO_ITEM_ID, ':', FROM_ITEM_ID) AS externalId, node_reference('PDM', TO_ITEM_ID) AS startNode, node_reference('PDM', FROM_ITEM_ID) AS endNodeFROM `src:001:avocet:db`.`ITEM_LINK` AS LINK, `src:001:avocet:db`.`avocet_001_pdm_structure_mapping` AS mapping, `src:001:avocet:db`.`ITEM` as ITEMWHERE ITEM.ITEM_ID = LINK.FROM_ITEM_ID AND ITEM.`ITEM_TYPE` = mapping.`source_entity_type` AND LINK.`LINK_TYPE` IN ( select concat( "'", replace(first(source_link_type), ",", "','"), "'" ) from `src:001:avocet:db`.`avocet_001_pdm_structure_mapping` AS strtcure_mapping, `src:001:avocet:db`.`ITEM` AS structure_ITEM where structure_ITEM.`ITEM_TYPE` = strtcure_mapping.`source_entity_type` ) and if directly used 'first_result','second_result' in IN Clause its giving desired result.
Hi @Viraj Dhanusha,I am currently using Cognite DB Extractor (dbextractor-winservice-v2.6.0)with version - v2.6.0and config file:version: 2logger: console: level: DEBUG file: level: DEBUG path: "file_2023.log"cognite: host: "https://westeurope-1.cognitedata.com" project: slb-pdf # (Either this or api-key is required) OIDC setup for CDF auth idp-authentication:# OIDC client ID client-id: ${COGNITE_CLIENT_ID}# OIDC client secret secret: ${COGNITE_CLIENT_SECRET}# URL to fetch OIDC tokens from token-url: "https://login.microsoftonline.com/45472a8e-6a12-4531-97d8-1f38f5d553a4/oauth2/v2.0/token"# List of OIDC scopes to request scopes: #- "https://api.cognitedata.com/.default" - "https://westeurope-1.cognitedata.com/.default" data-set: id: 1409796585286854 #extraction-pipeline: # external-id: "Historical_Test_TS_pipeline-ExtID"extractor:# (Optional) Number of rows to fetch from sources before uploading upload-queue-size: 10000# (Optional) Where to store extraction states (pr
@Dilini Fernando we have removed the duplicates externalIds from the transformation as suggested by @Anders Hafreager and the transformations are running smoothly.
Hi @Anders Hafreager is there any way we can ingest this data with same externalId, As have deleted the data from the model with transformation?And also we are not using same extrenalId twice in the transformation.
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.
Sorry, our virus scanner detected that this file isn't safe to download.
Check the
documentation
Ask the
Community
Take a look
at
Academy
Cognite
Status
Page
Contact
Cognite Support
