Skip to main content
Answer

ML model and training it

  • November 9, 2023
  • 8 replies
  • 112 views

Aditya Kotiyal
MVP
Forum|alt.badge.img+5

Hi Experts,

 

Few queries:

 

  1. Can I create a ML model directly using CDF functions? - I believe we can.
  2. If yes, how can I retrain a model without using any external AI platform? Can I use workflow orchestration to retrain a model with new data?

 

Thanks,

Aditya

Best answer by Jason Dressel

@Aditya Kotiyal ,

I worked with SWN team on this.  In short

  • Build/Train model externally → Push to CDF as a File
  • Within a Cognite Function, load the ML model and call it
  • Retraining can be via Cognite Function as well (within CDF Function limitations: memory, runtime) → push back to CDF as a File

Hope this helps,
-Jason

8 replies

Forum|alt.badge.img
  • Seasoned Practitioner
  • November 9, 2023

@Aditya Kotiyal ,

ML models are routinely saved and fetched to/from CDF files in Functions to enable end to end use cases..  Whether CDF Functions are appropriate to ‘train’ and ‘retrain’ depends on the use case as CDF Functions have finite resource limits (memory, cpu and runtime).  

-Jason

fyi @Anvar Akhiiartdinov @Andris Piebalgs @Chad Hutchison Can you advise further?


Aditya Kotiyal
MVP
Forum|alt.badge.img+5

Thanks @Jason Dressel .

Possible to share those resource limits?


Jørgen Lund
Seasoned Practitioner
Forum|alt.badge.img
  • Product Manager
  • November 13, 2023

@Aditya Kotiyal Cognite Functions resource limits can be found here.


Dilini Fernando
Seasoned Practitioner
Forum|alt.badge.img+2
  • Seasoned Practitioner
  • November 22, 2023

Hi @Aditya Kotiyal,

We are following up to see whether you're satisfied with the responses you've received.

If you found the responses offered by our community members to be instructive and helpful, we kindly request that you consider marking the most helpful response as the "Best Answer."

Best regards,
Dilini 

 


Aditya Kotiyal
MVP
Forum|alt.badge.img+5

Hi @Dilini Fernando ,

 

I was expecting other tagged by Jorgen to reply.

It will be good to understand how much CDF can be leveraged for AI/ML kind of workflows.


Dilini Fernando
Seasoned Practitioner
Forum|alt.badge.img+2
  • Seasoned Practitioner
  • November 30, 2023

Hi@Anvar Akhiiartdinov @Andris Piebalgs @Chad Hutchison

Please let us know your thoughts on this.

Br,

Dilini


Forum|alt.badge.img
  • Seasoned Practitioner
  • Answer
  • February 22, 2024

@Aditya Kotiyal ,

I worked with SWN team on this.  In short

  • Build/Train model externally → Push to CDF as a File
  • Within a Cognite Function, load the ML model and call it
  • Retraining can be via Cognite Function as well (within CDF Function limitations: memory, runtime) → push back to CDF as a File

Hope this helps,
-Jason


Aditya Kotiyal
MVP
Forum|alt.badge.img+5

Thanks @Jason Dressel