Migrating from Cognite Core Data Model to a Custom Extended Data Model

​@Nikola Vasiljevic will you be able to help out here?

/* MAPPING_MODE_ENABLED: false */
/* {"version":1,"sourceType":"fdm","mappings":[{"from":"externalId","to":"externalId","asType":"STRING"},{"from":"type","to":"_type","asType":"STRUCT<`space`:STRING, `externalId`:STRING>"},{"from":"name","to":"name","asType":"STRING"},{"from":"source","to":"source","asType":"STRUCT<`space`:STRING, `externalId`:STRING>"},{"from":"path","to":"path","asType":"ARRAY<STRUCT<`space`:STRING, `externalId`:STRING>>"},{"from":"root","to":"root","asType":"STRUCT<`space`:STRING, `externalId`:STRING>"},{"from":"description","to":"Description","asType":"STRING"},{"from":"assetClass","to":"assetClass","asType":"STRUCT<`space`:STRING, `externalId`:STRING>"},{"from":"sourceUpdatedUser","to":"sourceUpdatedUser","asType":"STRING"},{"from":"pathLastUpdatedTime","to":"pathLastUpdatedTime","asType":"TIMESTAMP"},{"from":"sourceUpdatedTime","to":"sourceUpdatedTime","asType":"TIMESTAMP"},{"from":"type","to":"type","asType":"STRUCT<`space`:STRING, `externalId`:STRING>"},{"from":"sourceCreatedUser","to":"sourceCreatedUser","asType":"STRING"},{"from":"aliases","to":"aliases","asType":"ARRAY<STRING>"},{"from":"","to":"simmonsPlantGUID","asType":"STRING"},{"from":"parent","to":"parent","asType":"STRUCT<`space`:STRING, `externalId`:STRING>"},{"from":"description","to":"description","asType":"STRING"},{"from":"tags","to":"tags","asType":"ARRAY<STRING>"},{"from":"sourceCreatedTime","to":"sourceCreatedTime","asType":"TIMESTAMP"},{"from":"sourceContext","to":"sourceContext","asType":"STRING"},{"from":"sourceId","to":"sourceId","asType":"STRING"},{"from":"object3D","to":"object3D","asType":"STRUCT<`space`:STRING, `externalId`:STRING>"}],"sourceLevel1":"cdf_cdm.CogniteCore.v1","sourceLevel2":"CogniteAsset"} */
select
  cast(`externalId` as STRING) as externalId,
  cast(
    `type` as STRUCT < `space`: STRING,
    `externalId`: STRING >
  ) as _type,
  cast(`name` as STRING) as name,
  cast(
    `source` as STRUCT < `space`: STRING,
    `externalId`: STRING >
  ) as source,
  cast(
    `path` as ARRAY < STRUCT < `space`: STRING,
    `externalId`: STRING > >
  ) as path,
  cast(
    `root` as STRUCT < `space`: STRING,
    `externalId`: STRING >
  ) as root,
  "Van Buren Cook Plant" as Description,
  cast(
    `assetClass` as STRUCT < `space`: STRING,
    `externalId`: STRING >
  ) as assetClass,
  cast(`sourceUpdatedUser` as STRING) as sourceUpdatedUser,
  cast(`pathLastUpdatedTime` as TIMESTAMP) as pathLastUpdatedTime,
  cast(`sourceUpdatedTime` as TIMESTAMP) as sourceUpdatedTime,
  cast(
    `type` as STRUCT < `space`: STRING,
    `externalId`: STRING >
  ) as type,
  cast(`sourceCreatedUser` as STRING) as sourceCreatedUser,
  cast(`aliases` as ARRAY < STRING >) as aliases,
  cast(
    `parent` as STRUCT < `space`: STRING,
    `externalId`: STRING >
  ) as parent,
  cast(`description` as STRING) as description,
  cast(`tags` as ARRAY < STRING >) as tags,
  cast(`sourceCreatedTime` as TIMESTAMP) as sourceCreatedTime,
  cast(`sourceContext` as STRING) as sourceContext,
  cast(`sourceId` as STRING) as sourceId,
  cast(
    `object3D` as STRUCT < `space`: STRING,
    `externalId`: STRING >
  ) as object3D
from
  cdf_data_models("cdf_cdm", "CogniteCore", "v1", "CogniteAsset")
where `externalId` = "Prepared_Foods_Van_Buren_Cook"

Here’s the SQL code I used for the transformation. 

​@Gerardo Chapa If I understand this question correctly, I would not call this migration. This seems to be how to extend CogniteAsset to your custom asset. It seems you have managed to do so well with the exception of one issue. In CogniteAsset, there are three special properties path, root, and pathLastUpdatedTime. This are READONLY as they are updated by a backend service based on the parent you set for each asset. So in you transformation, you are currently trying to write to these three fields, if you remove that, it should work fine.

Thanks Anders, I modified my transformation and I’m able to run it successfully, but I have some questions:

1. When I see the tree structure on the location I see the following:

but when clicking on one of the custom assets I see the following on the asset’s tree view:

Wondering if this is because the tree view only shows assets of the same type?

2. I originally created the some nodes as assets, but seems like I would need to transform them to equipment, could I easily transform from different types like derived from asset to derived from equipment?

This has to do with the Search UI. ​@Sofie Svartdal Berge will be better able to answer than I am.

Hi ​@Gerardo Chapa !

Wondering if this is because the tree view only shows assets of the same type?

I believe this is the reason for your blank Tree view, yes - that the defined “path” does not have nodes in the same view (Category) as the view you are opening the node through. 

We do have it on our backlog to support rendering the Tree view through the foundational “CogniteAsset” view, but I do not have a clear timeline for this yet. 

Thanks ​@Sofie Svartdal Berge, is there anyone else that can answer my second question? 

​@Gerardo Chapa  I am a bit confused by you question. I would assume you have a transformations that targets assets. Can’t you update that to target equipment instead?

Thanks for your reply ​@Anders Albert , sorry if I was not clear enough, what I meant is I have a set of CogniteAssets, that I need to transform into CustomEquipment that extends CogniteEquipment. The Custom Data Model we built some of the existing CogniteAssets better match the custom equipment, rather than the CogniteAsset, would transforming the CogniteAssets into CustomEquipment duplicate the nodes, or does the transformation turns the existing CogniteAssets into CustomEquipment?

No, it will become both. All instances in data modeling are nodes or edges. A node is a CogniteAsset if it has properties in the CogniteAsset storage and similar if it has properties in the CogniteEquipment stoarge it is considered equipment. So if you create a transformation that populate the CogniteEquipment from CogniteAsset it will become both. As far as I know, you would have to delete and recreate to ensure it is only a CogniteEquipment.