Skip to main content
Solved

Connect Cognite function to private end point Azure key vault to store confidential information


Hi Cognite Team,

Is there any document which can show , how to fetch confidential information from private end point key vault  on Microsoft azure key vault.

 

Best answer by Mithila Jayalath

@sarojbala please refer to the documentation here.

View original
Did this topic help you find an answer to your question?

9 replies

Forum|alt.badge.img+1

Hi Saroj, 
 

It’s Viraj here.
It is possible to connect to the Azure key vault using Cognite functions. You can refer to this documentation for more information. Let me know if you need any further information. 
 

Best regards, 
Viraj 


  • Author
  • Committed
  • 4 replies
  • August 31, 2023

This is generic Microsoft document. is there any repo provided by CDF which can show example to connect to key vault from cognite function


Hello Saroj!

Just so we can better understand your use case, you want the Cognite Function to be able to fetch confidential information from an Azure Key Vault when executing (for instance an API-key or a secret token)?
 

Unfortunately, we do not have any Cognite specific documentation on how to do this, but you can certainly do this using the documentation Viraj linked to as well as the `azure.keyvault` and `azure-keyvault-secrets` SDK. 

Make sure you deploy your Cognite Function with the `azure-identity` and `azure-keyvault-secrets` package listed in the `requirements.txt`. Then within your handler, you can authenticate by various means. I recommend for instance the `ClientSecretCredential` method, but you would have to tailor this to your use case.

 

Then within your handler, you can fetch the secret as such:

from azure.keyvault.secrets import SecretClient
from azure.identity import ... # credential type of choice

def handle(client, data):

    keyVaultName = data.get("keyvault_name")
    KVUri = f"https://{keyVaultName}.vault.azure.net"

    credential = ... # use your credentials of choice here. 
    client = SecretClient(vault_url=KVUri, credential=credential)

    retrieved_secret = client.get_secret(secretName)


If you need client-ID and client-secret in order to authenticate against Azure, you can for instance deploy your Cognite function with these set in the `secrets`-field. 


  • Author
  • Committed
  • 4 replies
  • September 20, 2023

Hi Ivar, I tried above approach and deployed a test cognite function.

Key vault is private end point key vault for us. Function is giving below error:

File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/keyvault/secrets/_client.py", line 72, in get_secret
bundle = self._client.get_secret(
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/keyvault/secrets/_generated/_operations_mixin.py", line 1640, in get_secret
return mixin_instance.get_secret(vault_base_url, secret_name, secret_version, **kwargs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
return func(*args, **kwargs)
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/keyvault/secrets/_generated/v7_4/operations/_key_vault_client_operations.py", line 767, in get_secret
map_error(status_code=response.status_code, response=response, error_map=error_map)
File "/home/site/wwwroot/.python_packages/lib/site-packages/azure/core/exceptions.py", line 165, in map_error
raise error
azure.core.exceptions.ClientAuthenticationError: (Unauthorized) AKV10032: Invalid issuer. Expected one of https://sts.windows.net/7a3c88ff-a5f6-449d-ac6d-e8e3aa508e37/, https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/, https://sts.windows.net/e2d54eb5-3869-4f70-8578-dee5fc7331f4/, found https://sts.windows.net/a9ae5b54-3600-4917-a9dc-3020723360b3/.
Code: Unauthorized
Message: AKV10032: Invalid issuer. Expected one of https://sts.windows.net/7a3c88ff-a5f6-449d-ac6d-e8e3aa508e37/, https://sts.windows.net/f8cdef31-a31e-4b4a-93e4-5f571e91255a/, https://sts.windows.net/e2d54eb5-3869-4f70-8578-dee5fc7331f4/, found https://sts.windows.net/a9ae5b54-3600-4917-a9dc-3020723360b3/.

 

Also providing code snippet

def handle(client, data):    print("<--Welcome to Bently daily count function-->")    KEYVAULT_NAME = "kv-abc-01"    KEYVAULT_URI = f"https://{KEYVAULT_NAME}.vault.azure.net"    _credential = DefaultAzureCredential()    _sc = SecretClient(vault_url=KEYVAULT_URI, credential=_credential, connection_verify=False)    CLIENT_ID = _sc.get_secret("kvSCRT-d-ussc-01").value

  • Practitioner
  • 20 replies
  • September 22, 2023

Hello again! 
 

Probable Cause: It seems you're using DefaultAzureCredential(), which can pull credentials from various sources (like environment variables, managed identities, etc.). If these sources aren't configured correctly, it can lead to such authentication issues.

Suggested Solution: To address this, I recommend trying the ClientSecretCredential method, which allows for more specific authentication using your Azure AD tenant ID, client ID, and client secret. Here's a basic example of how to use it:

def handle(client, data, secrets):

    tenant_id = "YOUR_TENANT_ID"
    client_id = "YOUR_CLIENT_ID"
    client_secret = secrets.get("YOUR_CLIENT_SECRET")

    credential = ClientSecretCredential(tenant_id=tenant_id, client_id=client_id, 
    client_secret=client_secret)

    KEYVAULT_NAME = "kv-abc-01"
    KEYVAULT_URI = f"https://{KEYVAULT_NAME}.vault.azure.net"
    sc = SecretClient(vault_url=KEYVAULT_URI, credential=credential)

    CLIENT_ID = sc.get_secret("kvSCRT-d-ussc-01").value

You would have to add the `YOUR_CLIENT_SECRET` as a secret when deploying the Cognite Function, as described here: https://docs.cognite.com/cdf/functions/#additional-arguments

Hope this helps! 


  • Author
  • Committed
  • 4 replies
  • September 25, 2023

Key vault is private end point key vault as shown below: How to link azure AAD app to this key vault so that i can use client id, secret using ClientSecretCredential method. Will i be allowed to request client to add App registration as access policy for private end point key vault.

 


  • Practitioner
  • 20 replies
  • September 25, 2023

Hello again!

This is slightly outside my area of expertise, but I will do my best to answer.
 

The "Disable public access" setting in Azure Key Vault is a security measure designed to restrict all public network access. In our context, this means that any request coming from a Cognite Function, which falls outside the allowed IP addresses or virtual networks, will be blocked, regardless of the authentication method.

Given this, there isn't a straightforward method to bypass this setting using any authentication approach, such as ClientSecretCredential. The network-level restriction set by "Disable public access" will always take precedence.

While I understand this is far from ideal, you might need to deactivate the “Disable public access” to make this work. Before making any changes, it's essential to consider the security implications.

Kind regards,
Ivar


Mithila Jayalath
Seasoned Practitioner
Forum|alt.badge.img
  • Seasoned Practitioner
  • 285 replies
  • Answer
  • September 25, 2023

@sarojbala please refer to the documentation here.


@sarojbala @Ivar Stangeby It is actually possible to do this, but it would require Cognite to create a private endpoint in their Azure tenant (granted that the CDF cluster is hosted on Azure). I would be interested in using private endpoints to access private Azure resources from CDF/cognite infra as well, so please update this thread if you are able to work it out :)

 

https://learn.microsoft.com/en-us/azure/architecture/guide/networking/cross-tenant-secure-access-private-endpoints


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings