Skip to main content
Question

how can we deploy cdf functions from azure devops using toolkit


Forum|alt.badge.img+2

Hi All,

I need to deploy CDF Functions from azure devops using cdf toolkit.

can you share me any example pipeline files which i can take it as reference.

 

Regards,

Nidhi N G

3 replies

  • Architect
  • 10 replies
  • September 4, 2024

Hi, this is a simple example of a pipeline file. A more thorough writeup will be forthcoming, but I hope this can help you get started: 

 

trigger:
- main

pool:
  vmImage: 'ubuntu-latest'

jobs:
- job: BuildAndDeploy
  displayName: 'Build and Deploy Modules'
  steps:
  - checkout: self
  - task: UsePythonVersion@0
    inputs:
      versionSpec: '3.11'
  - script: |
      curl -sSL https://install.python-poetry.org | python3.11 -
    displayName: 'Install Poetry'
  - script: |
      poetry install
    displayName: 'Install Dependencies'
  - script: |
      poetry run cdf-tk build --env=dev --clean
    displayName: 'Build the Package'
    workingDirectory: './new-project'
    env: 
      CDF_CLUSTER: '<cluster>'
      CDF_PROJECT: '<project>'
      IDP_CLIENT_ID: '<client-id>'
      IDP_CLIENT_SECRET: $(IDP_CLIENT_SECRET_MASTER)
      IDP_TOKEN_URL: 'https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/token'
      TRANSFORMATIONS_CLIENT_ID: '<transformations-client-id>'
      TRANSFORMATIONS_CLIENT_SECRET: $(TRANSFORMATIONS_CLIENT_SECRET_MASTER)
      FUNCTIONS_CLIENT_ID: '<function-client-id>'
      FUNCTIONS_CLIENT_SECRET: $(FUNCTIONS_CLIENT_SECRET_MASTER)
  - script: |
      poetry run cdf-tk deploy --drop --env=dev
    displayName: 'Deploy the Package'
    workingDirectory: './new-project'
    env:
      CDF_CLUSTER: '<cluster>'
      CDF_PROJECT: '<project>'
      IDP_CLIENT_ID: '<client-id>'
      IDP_CLIENT_SECRET: $(IDP_CLIENT_SECRET_MASTER)
      IDP_TOKEN_URL: 'https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/token'
      TRANSFORMATIONS_CLIENT_ID: '<transformations-client-id>'
      TRANSFORMATIONS_CLIENT_SECRET: $(TRANSFORMATIONS_CLIENT_SECRET_MASTER)
      FUNCTIONS_CLIENT_ID: '<function-client-id>'
      FUNCTIONS_CLIENT_SECRET: $(FUNCTIONS_CLIENT_SECRET_MASTER)

 


Forum|alt.badge.img+2
  • Author
  • Seasoned
  • 3 replies
  • September 6, 2024

Hi @palronning  ,

thank you for the information.

need to know more about this pipeline file, 

  1. where should i put my functions folder in the repo which is having handler.py
  2. how we give the functions path here in this pipeline and make sure when this pipeline is run, the functions get deployed.
  3. how the folder structure in the azure repo should look like in for functions deployment.

can you please try to help me here for my above questions.

thank you

Nidhi 


  • Architect
  • 10 replies
  • September 9, 2024

Hi @N G Nidhi , I think I understand your question better now. 

The Toolkit treats functions as any other resource type, hence they need to be added to a folder like this:

modules
    my_module
        auth
        functions
        .... 


and be selected in the config.dev.yaml:

selected:
   modules/my_module


The in the pipeline above, the workingDirectory: './new-project' simply has to point to the parent directory of modules/ and config.dev.yaml. 

The required format for functions is described here: https://docs.cognite.com/cdf/deploy/cdf_toolkit/references/configs#functions 

You will see that you need a yaml file, an optional schedule yaml file and a subdirectory for the function code itself. 

As of now, development and testing support for functions is a bit limited in the Toolkit. We hope to improve that in future versions.


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings