Two of the important goals with a Data Mesh Architecture


Two of the important goals with a Data Mesh Architecture would be:

  1. Ensure that the users of data can easily find and trust the data – through carefully consider distribution of ownership and governance throughout the company/domains
  2. Ensure that data is “interoperable” across domains – to understand the meaning of data from one domain in the context of another domain.

Question: What are the challenges and advantages of a Data Mesh Architecture to achieve these goals? E.g.:

  1. Domain expert knowledge and capacity within the business area/domain vs centralized knowledge
  2. Ability to make data interoperable across domains vs all-inclusive master data management
  3. Make it easier for end users to make use of the data
  4. Change management – move towards a distributed data ownership model where ownership is understood and prioritized day-to-day.

Other thoughts? 👀


2 replies

I see Data Mesh as a new and different approach to meet the needs you lay out as the important goals; easy access to trustworthy data I can understand and use. These, I would say, are some of the key goals of any approach for how to architect your data and your whole. Data Mesh, as coined by Zhamak, is according to my understanding, a way to achieve these and other goals while ensuring your system maintains and can be easily changed, and where data is more updated. Like the move we have observed from huge monolithic applications to a micro services approach for the way applications that need frequent updates and changes are designed today. This is of course a continuous balance between ability for easy and quick change on one side (often decentralization apporach), and a way to ensure cost efficiency on the other (often centralization apporach).

If a company needs to go the Mesh Data rout or down the more known central data management highway, depends on the business model, the business needs and priorities. It will also depend on how complex and diverse the business model is. Both approaches will have it’s struggles. The central approach depends on a heavy ETL-setup and sometimes complex synchronization mechanisms that eventually make changes moder comples and timeconsuming, whereas a decentral approach can suffer from missed opportunities if data is not set up for sharing and if discoverability is low, and the ETL-problems are still present but more fragmented, but are believed to support some of the frequent changes in a better way. Data Mesh being a new approach, all pros and cons are obviously not fully understood, and early adopters we need to apply caution. Every company need to make up their minds and do proper business case and strategic analyses. Different approaches to data architecture and management will obviously require different setups for optimizing domain data expertise, different governance models, roles and responsibilities.  Hence, one governance setup for one company can be wrong for another if they have decided on a different data architecture.

Thank you for very thoughtful deliberations on the topic of Data Mesh. I certainly agree that the concept is a new approach, where there are many pros and cons that are not fully understood. 

I would be very curious to explore a few topics further, amongst others:

  • How can one get a bit of “Ole Brum - ja takk begge deler” when it comes to a decentralized governance vs. centralized best practices and ability to create real interoperability between domains and data products?
  • Does enabling real availability and discoverability of data for consumption broadly in an organization always come with the price of a long, heavy and expensive processes?

Very happy to continue the discussion in future foras! 

Reply