Connect Cognite function to private end point Azure key vault to store confidential information

Hi Saroj, 
 

It’s Viraj here.
It is possible to connect to the Azure key vault using Cognite functions. You can refer to this documentation for more information. Let me know if you need any further information. 
 

Best regards, 
Viraj 

This is generic Microsoft document. is there any repo provided by CDF which can show example to connect to key vault from cognite function

Hello Saroj!

Just so we can better understand your use case, you want the Cognite Function to be able to fetch confidential information from an Azure Key Vault when executing (for instance an API-key or a secret token)?
 

Unfortunately, we do not have any Cognite specific documentation on how to do this, but you can certainly do this using the documentation Viraj linked to as well as the `azure.keyvault` and `azure-keyvault-secrets` SDK. 

Make sure you deploy your Cognite Function with the `azure-identity` and `azure-keyvault-secrets` package listed in the `requirements.txt`. Then within your handler, you can authenticate by various means. I recommend for instance the `ClientSecretCredential` method, but you would have to tailor this to your use case.

Then within your handler, you can fetch the secret as such:

from azure.keyvault.secrets import SecretClient
from azure.identity import ... # credential type of choice

def handle(client, data):

    keyVaultName = data.get("keyvault_name")
    KVUri = f"https://{keyVaultName}.vault.azure.net"

    credential = ... # use your credentials of choice here. 
    client = SecretClient(vault_url=KVUri, credential=credential)

    retrieved_secret = client.get_secret(secretName)

If you need client-ID and client-secret in order to authenticate against Azure, you can for instance deploy your Cognite function with these set in the `secrets`-field. 

Hi Ivar, I tried above approach and deployed a test cognite function.

Key vault is private end point key vault for us. Function is giving below error:

File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/keyvault/secrets/_client.py", line 72, in get_secret
bundle = self._client.get_secret(
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/keyvault/secrets/_generated/_operations_mixin.py", line 1640, in get_secret
return mixin_instance.get_secret(vault_base_url, secret_name, secret_version, **kwargs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/keyvault/secrets/_generated/v7_4/operations/_key_vault_client_operations.py", line 767, in get_secret
map_error(status_code=response.status_code, response=response, error_map=error_map)
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/core/exceptions.py", line 165, in map_error
raise error
azure.core.exceptions.ClientAuthenticationError: (Unauthorized) AKV10032: Invalid issuer. Expected one of https://sts.windows.net/7a3c88ff-a5f6-449d-ac6d-e8e3aa508e37/, https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/, https://sts.windows.net/e2d54eb5-3869-4f70-8578-dee5fc7331f4/, found https://sts.windows.net/a9ae5b54-3600-4917-a9dc-3020723360b3/.
Code: Unauthorized
Message: AKV10032: Invalid issuer. Expected one of https://sts.windows.net/7a3c88ff-a5f6-449d-ac6d-e8e3aa508e37/, https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/, https://sts.windows.net/e2d54eb5-3869-4f70-8578-dee5fc7331f4/, found https://sts.windows.net/a9ae5b54-3600-4917-a9dc-3020723360b3/.

Also providing code snippet

def handle(client, data):    print("<--Welcome to Bently daily count function-->")    KEYVAULT_NAME = "kv-abc-01"    KEYVAULT_URI = f"https://{KEYVAULT_NAME}.vault.azure.net"    _credential = DefaultAzureCredential()    _sc = SecretClient(vault_url=KEYVAULT_URI, credential=_credential, connection_verify=False)    CLIENT_ID = _sc.get_secret("kvSCRT-d-ussc-01").value

Hello again! 
 

Probable Cause: It seems you're using DefaultAzureCredential(), which can pull credentials from various sources (like environment variables, managed identities, etc.). If these sources aren't configured correctly, it can lead to such authentication issues.

Suggested Solution: To address this, I recommend trying the ClientSecretCredential method, which allows for more specific authentication using your Azure AD tenant ID, client ID, and client secret. Here's a basic example of how to use it:

def handle(client, data, secrets):

    tenant_id = "YOUR_TENANT_ID"
    client_id = "YOUR_CLIENT_ID"
    client_secret = secrets.get("YOUR_CLIENT_SECRET")

    credential = ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, 
    client_secret=client_secret)

    KEYVAULT_NAME = "kv-abc-01"
    KEYVAULT_URI = f"https://{KEYVAULT_NAME}.vault.azure.net"
    sc = SecretClient(vault_url=KEYVAULT_URI, credential=credential)

    CLIENT_ID = sc.get_secret("kvSCRT-d-ussc-01").value

You would have to add the `YOUR_CLIENT_SECRET` as a secret when deploying the Cognite Function, as described here: https://docs.cognite.com/cdf/functions/#additional-arguments

Hope this helps! 

Key vault is private end point key vault as shown below: How to link azure AAD app to this key vault so that i can use client id, secret using ClientSecretCredential method. Will i be allowed to request client to add App registration as access policy for private end point key vault.

Hello again!

This is slightly outside my area of expertise, but I will do my best to answer.
 

The "Disable public access" setting in Azure Key Vault is a security measure designed to restrict all public network access. In our context, this means that any request coming from a Cognite Function, which falls outside the allowed IP addresses or virtual networks, will be blocked, regardless of the authentication method.

Given this, there isn't a straightforward method to bypass this setting using any authentication approach, such as ClientSecretCredential. The network-level restriction set by "Disable public access" will always take precedence.

While I understand this is far from ideal, you might need to deactivate the “Disable public access” to make this work. Before making any changes, it's essential to consider the security implications.

Kind regards,
Ivar

@sarojbala @Ivar Stangeby It is actually possible to do this, but it would require Cognite to create a private endpoint in their Azure tenant (granted that the CDF cluster is hosted on Azure). I would be interested in using private endpoints to access private Azure resources from CDF/cognite infra as well, so please update this thread if you are able to work it out :)

https://learn.microsoft.com/en-us/azure/architecture/guide/networking/cross-tenant-secure-access-private-endpoints