Question

End-to-end ML Models Management

  • 13 May 2024
  • 2 replies
  • 34 views
Andre Alves
Rank
Userlevel 4
Badge +6

Are there any documented use cases or papers on integrating MLflow with Cognite, or is it something we need to implement ourselves?

For example, if we aim to seamlessly integrate the MLflow UI with Cognite to evaluate and select the top-performing models, we could leverage SQLite, which operates on the local file system (e.g., mlruns.db) and provides a built-in client, sqlite3. However, our preference is to seamlessly integrate it with Cognite.

André Alves

2 replies

K
Rank
Userlevel 3
Badge

Hi, Andre.

Thank you for reaching out on this topic. 

We do not have an out-of-the-box integration with MLflow productized, but our delivery team has make plugin that is in a lower maturity phase that we could get in touch with you and see if it would be helpful for you.

If that is interesting, we can reach out through our partner teams and get the knowledge sharing started.

 

Best regards

Knut Vidvei, Cognite

 

Andre Alves
Rank
Userlevel 4
Badge +6

Thank you, @Knut Vidvei ,

We are designing a solution to bring MLOps value to a potential client, and our proposed approach is as follows:

  1. Use Git for version control: All pipelines and code will be stored in Git to ensure robust version control and collaborative development.
  2. Store data in Cognite CDF: Data will be efficiently and scalably stored in Cognite CDF.
  3. Manage model development with MLflow: MLflow will be used to streamline and track the model development process, ensuring reproducibility and manageability.
  4. Manage the model lifecycle with a hybrid solution using Cognite and Azure services: The model lifecycle, from development to deployment, will be managed through an integrated solution combining Cognite and Azure services.

We would greatly appreciate any insights or experiences that partner teams can share regarding similar implementations.

Thanks, André

André Alves

Reply


Terms & ConditionsCookie Settings