Skip to main content
Solved

Issues with asset transformation code

  • November 26, 2024
  • 4 replies
  • 39 views

In working with CDF: Integrate, I am trying to transform the asset csv file using the following code: 

 

SELECT
concat('Ruby2001:',loc) as externalId,
IF(parent_loc='' OR parent_loc IS NULL, '', concat('Ruby2001:',parent_loc)) AS parentExternalId,

CAST(lastUpdatedTime AS STRING) AS name,

to_metadata(*) AS metadata,

description AS description,

2869936105026304 AS dataSetId
 
FROM `Ruby2001`.assets

 

However I keep getting this error message: 

Column 'parent_loc' does not exist. Did you mean one of the following? [z6eecbaa9603c4ee195055fe6aaeb89a3.loc, z6eecbaa9603c4ee195055fe6aaeb89a3.key, z6eecbaa9603c4ee195055fe6aaeb89a3.name, z6eecbaa9603c4ee195055fe6aaeb89a3.parent_loc,,, z6eecbaa9603c4ee195055fe6aaeb89a3.WMT_TAG_ID, z6eecbaa9603c4ee195055fe6aaeb89a3.description, z6eecbaa9603c4ee195055fe6aaeb89a3.WMT_SYSTEM_ID, z6eecbaa9603c4ee195055fe6aaeb89a3.lastUpdatedTime]; line 3 pos 3; 'Project [concat(Ruby2001:, loc#41009138) AS externalId#41009101, 'IF((('parent_loc = ) OR isnull('parent_loc)), , 'concat(Ruby2001:, 'parent_loc)) AS parentExternalId#41009102, cast(lastUpdatedTime#41009134 as string) AS name#41009103, map_from_arrays(array(key, lastUpdatedTime, WMT_SYSTEM_ID, WMT_TAG_ID, description, loc, name, parent_loc,,), array(cast(key#41009133 as string), cast(lastUpdatedTime#41009134 as string), cast(WMT_SYSTEM_ID#41009135 as string), cast(WMT_TAG_ID#41009136 as string), cast(description#41009137 as string), cast(loc#41009138 as string), cast(name#41009139 as string), cast(parent_loc,,#41009140 as string))) AS metadata#41009104, description#41009137 AS description#41009105, 2869936105026304 AS dataSetId#41009106L] +- SubqueryAlias z6eecbaa9603c4ee195055fe6aaeb89a3 +- View (`z6eecbaa9603c4ee195055fe6aaeb89a3`, [key#41009133,lastUpdatedTime#41009134,WMT_SYSTEM_ID#41009135,WMT_TAG_ID#41009136,description#41009137,loc#41009138,name#41009139,parent_loc,,#41009140]) +- Relation [key#41009133,lastUpdatedTime#41009134,WMT_SYSTEM_ID#41009135,WMT_TAG_ID#41009136,description#41009137,loc#41009138,name#41009139,parent_loc,,#41009140] RawTableRelation(db=Ruby2001, table=assets) | code: 400 | X-Request-ID: ce89d63f-1c78-97d2-9165-88b8621e92dd

 

I have checked the original asset csv and the variables are there but separated by a ‘;’ - should they be separated by ‘,’ instead? Thanks 

Best answer by Sofie Haug

Hi ​@rubyhaddad_01  I had a look at your CSV file, and I’m not sure, but somethings seems a bit strange. The loc column and all it’s values have had some extra commas added at the end. Can you try to delete the raw table and uplaod the file again? 

View original
Did this topic help you find an answer to your question?

Jan Inge Bergseth
MVP

I would recommend to split this in 2 Transformations.

First just creating all the Assets, then as second Transformation create the connection  with parent_loc


Jan Inge Bergseth
MVP

with 2 transformations you make sure the “parent” node is written to the backend before you try to connect to it.

But required that the data actually contain the parent so the child's have a parent 


Sofie Haug
Seasoned Practitioner
Forum|alt.badge.img+7
  • Cognite Academy Instructor
  • November 26, 2024

Hi ​@rubyhaddad_01  I had a look at your CSV file, and I’m not sure, but somethings seems a bit strange. The loc column and all it’s values have had some extra commas added at the end. Can you try to delete the raw table and uplaod the file again? 


Sofie Haug
Seasoned Practitioner
Forum|alt.badge.img+7
  • Cognite Academy Instructor
  • November 26, 2024

I also tried to read form a diffent table, and then it was all good! So nothing wrong with the SQL code part, just the RAW table itself.

 

 


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings