Solved

recommendations for managing/decomposing complex models

  • 16 December 2022
  • 2 replies
  • 52 views
Juan Alvarado
Rank
Userlevel 3
Badge

As we develop richer models supporting a greater variety of workflows, we have started asking questions about model maintenance and modularization. Decomposing large models into sub-models that have low coupling seems attractive, but would require some type of referential support across models. Is Cognite considering such a feature, or how do we envision managing complex models in general? As you are probably aware large models have an increased risk that a breaking change in a peripheral type will force a new version on the entire model. Since data must be manually migrated today, this creates a maintenance load that we would be keen to address.

icon

Best answer by Anders Hafreager 16 December 2022, 10:39

View original

2 replies

T
Rank
Userlevel 3

Hi Juan, and thank you for this considered expectation.

 

We’re thinking of the initial release of FDM as the starting point for the service. To us, that means we still have some difficult problems to resolve and evolve the functionality after the GA release, including the one of how to handle model changes and their implications with a more robust suite of tools. 
 

The team is excited to work with you and others at mapping out opportunities, solutions and needs in this space!

Anders Hafreager
Rank
Userlevel 3

I’ll shine some more light on this topic as well. FDM is designed to reduce the amount of data migration needed on non breaking data types even though the data model itself has a breaking change. One example would be that you remove a property from a type (leading to a breaking change). In this scenario, there is no need for any data migration as the other properties still point to the same physical storage. 

An example of a breaking change where data needs to be migrated is restructuring of physical storage, and change of data types (int → float for example). Here we have plans for making this a joyful experience later on, but at GA, the recommended way to solve this is create/update a CDF transformation that will handle it. 

I am very interested in having a session on what kind of breaking changes you expect so we ensure they are covered greatly from the product.

Reply


Terms & ConditionsCookie Settings