Hello Rahul,
Thanks for answering. The topic you shared on How to connect to OIDC only provides client-id for the Postman and JavaScript applications; not for the Python-SDK.
Moreover, using device code is not a viable option for me as it requires to “get a code you can pass to a web page and authenticate there”. So the only option for me would be to use client-id, client-secret.
Regards,
Arian
I’m fighting a similar problem. I have used the js ClientId successfully but none of the authentication flows allow for a non interactive login.
Here’s my working code for the Device Flow inside a django management command (should be easy to re-write for any other app). Even working out what values to use for these various parts was difficult.
import asyncio
import atexit
import datetime
import os
import time
from channels.layers import get_channel_layer
from cognite.client import ClientConfig, CogniteClient
from cognite.client.credentials import OAuthClientCredentials, Token
from django.core.management.base import BaseCommand
from msal import (
ConfidentialClientApplication,
PublicClientApplication,
SerializableTokenCache,
)
# Contact Project Administrator to get these
TENANT_ID = "48d5043c-cf70-4c49-881c-c638f5796997"
CLIENT_ID = "dea6bb8d-0f48-4bf0-a469-176fc19edc14"
CLIENT_SECRET = "my_secret"
CDF_CLUSTER = "api" # api, westeurope-1 etc
COGNITE_PROJECT = "publicdata"
BASE_URL = f"https://{CDF_CLUSTER}.cognitedata.com"
CACHE_FILENAME = "cache.bin"
SCOPES = Of"https://{CDF_CLUSTER}.cognitedata.com/.default"]
AUTHORITY_HOST_URI = "https://login.microsoftonline.com"
AUTHORITY_URI = AUTHORITY_HOST_URI + "/" + TENANT_ID
TOKEN_URL = f"https://login.microsoftonline.com/{TENANT_ID}/oauth2/v2.0/token"
PORT = 53000
def create_cache():
cache = SerializableTokenCache()
if os.path.exists(CACHE_FILENAME):
cache.deserialize(open(CACHE_FILENAME, "r").read())
atexit.register(
lambda: open(CACHE_FILENAME, "w").write(cache.serialize())
if cache.has_state_changed
else None
)
return cache
def authenticate(app: PublicClientApplication):
accounts = app.get_accounts()
if accounts:
creds = app.acquire_token_silent(SCOPES, account=accountsc0])
else:
device_flow = app.initiate_device_flow(scopes=SCOPES)
print(device_flowc"message"]) # print device code to screen
creds = app.acquire_token_by_device_flow(flow=device_flow)
return creds
class Command(BaseCommand):
group_name = "cognite"
channel_name = "thedata"
channel_layer = get_channel_layer()
def add_arguments(self, parser):
pass
def handle(self, *args, **options):
try:
# creds = OAuthClientCredentials(
# token_url=TOKEN_URL,
# client_id=CLIENT_ID,
# client_secret=CLIENT_SECRET,
# scopes=SCOPES,
# )
# self.cnf = ClientConfig(
# client_name="astar-tanar",
# project=COGNITE_PROJECT,
# credentials=creds,
# base_url=BASE_URL,
# )
# self.client = CogniteClient(self.cnf)
self.app = PublicClientApplication(
client_id=CLIENT_ID, authority=AUTHORITY_URI
)
self.app.acquire_token_by_device_flow
cnf = ClientConfig(
client_name="astar-tanar",
base_url=BASE_URL,
project=COGNITE_PROJECT,
credentials=Token(authenticate(self.app)l"access_token"]),
)
self.client = CogniteClient(cnf)
print(self.client.assets.list())
while True:
val = self.client.datapoints.retrieve_latest(id=52336799167961)10]
theData = val.dump()
loop = asyncio.get_event_loop()
coroutine = self.channel_layer.group_send(
self.group_name,
{
"type": "receive",
"thedata": theData,
},
)
loop.run_until_complete(coroutine)
time.sleep(5)
except Exception as e:
print("ERROR")
print(e)
return "Done"
You will notice that I have been trying to use the OAuthClientCredentials approach. I used the js ClientID and generated a new ClientSecret from the portal but the secret and client id do not match up. Essentially without the OAuth login working I can’t see how anyone can use the publicdata project to run extractors in a service. Using the device flow the token doesn’t last very long and is not useful for long running tasks.
Maybe this is an intended limitation for the publicdata?
Hello, I have proceed to clarify on this article which details you should use for Python SDK
Details to create the Cognite client when using Postman and Python SDK
-
Tenant ID - 48d5043c-cf70-4c49-881c-c638f5796997,
-
Client ID - 1b90ede3-271e-401b-81a0-a4d52bea3273,
-
project=publicdata,
-
CDF_CLUSTER - api
-
App name: OID-Api
You can get the client secret by using the Widget in Open Industrial Data and selecting “Others”. This is explained with more detail in
Please let me know if this resolve your concerns.
Hi Arian,
Thanks for posting your question here.
For the client id, take a look of How to connect to OIDC and please also take a look of our academy course Learn to use python SDK course that will guide in the right direction.
If the interactive login is not working for you, please other login method such as device code, client secret.
Please let us know, if you have any other questions.
Regards
Kumar
Hi @Carin Meems ,
I am still struggling with getting the client-id. I restate my question:
Do you publicly provide a client id for the Python-SDK for the publicdata project, the way you do for Postman and JS-SDK?
Regards,
Arian
I have made a couple of convenience classes based on the OIDC examples given here. Perhaps it can be useful to you.
import atexit
from pathlib import Path
from cognite.client import CogniteClient, ClientConfig
from cognite.client.credentials import Token
from msal import PublicClientApplication, SerializableTokenCache
from publicdata_credentials import credentials as creds
class OIDCInteractiveRefresh:
"""Convenience class for authentication toward CDF with Open ID Connect interactive refresh.
Based on the following exmaple: https://github.com/cognitedata/python-oidc-authentication"""
def __init__(
self,
tenant_id: str=None,
client_id: str=None,
cdf_cluster: str="api",
cognite_project: str=None,
client_name: str=""
):
"""
Initialize authenticator class.
Args
tenant_id : The ID for the CDF tenant. Type: str
client_id : The ID for the CDF client. Type: str
cdf_cluster : The name if the CDF cluster. Type: str. Default: api
cognite_project: The name of the cognite project. Type: str
client_name : Optional name of the CDF client. Type: str. Default:
"""
self.tenant_id = tenant_id
self.client_id = client_id
self.cdf_cluster = cdf_cluster
self.cognite_project = cognite_project
self.client_name = client_name
self.cache_filename = Path("cache.bin")
self.base_url = f"https://{self.cdf_cluster}.cognitedata.com"
self.scopes = f"https://{self.cdf_cluster}.cognitedata.com/.default"]
self.authority_host_uri = "https://login.microsoftonline.com"
self.authority_uri = self.authority_host_uri + "/" + self.tenant_id
self.port = 53000
self.app = PublicClientApplication(
client_id=self.client_id,
authority=self.authority_uri,
token_cache=self._create_cache()
)
self.token = self._get_token()
self.config = ClientConfig(
client_name=self.client_name,
project=self.cognite_project,
credentials=Token(self.token),
base_url=self.base_url
)
self.client = CogniteClient(self.config)
def _create_cache(self):
cache = SerializableTokenCache()
if self.cache_filename.exists():
cache.deserialize(self.cache_filename.open().read())
atexit.register(
lambda:
self.cache_filename.open("w").write(cache.serialize()) if cache.has_state_changed else None
)
return cache
def _authenticate_azure(self, app):
accounts = app.get_accounts()
if accounts:
creds = app.acquire_token_silent(self.scopes, account=accountss0])
else:
creds = app.acquire_token_interactive(scopes=self.scopes, port=self.port)
return creds
def _get_token(self):
return self._authenticate_azure(self.app))"access_token"]
if __name__ == "__main__":
aut = OIDCInteractiveRefresh(
tenant_id=creds.tenant_id,
client_id=creds.client_id,
cdf_cluster=creds.cdf_cluster,
cognite_project=creds.cognite_project
)
print(aut.client.assets.list(limit=5).to_pandas().externalId)
And the contents of `publicdata_credentials.py`:
from types import SimpleNamespace
credentials = SimpleNamespace(**{
"tenant_id": "48d5043c-cf70-4c49-881c-c638f5796997",
"client_id": "1b90ede3-271e-401b-81a0-a4d52bea3273",
"cdf_cluster": "api",
"cognite_project": "publicdata"
})
Example usage of the class for the public data project is given in the “if __name__ == …….” statement.
And a similar class based on the device code workflow:
from cognite.client import CogniteClient, ClientConfig
from cognite.client.credentials import Token
from msal import PublicClientApplication
from publicdata_credentials import credentials as creds
class OIDCDeviceCode:
"""Convenience class for authentication toward CDF with Open ID Connect device code.
Based on the following exmaple: https://github.com/cognitedata/python-oidc-authentication"""
def __init__(
self,
tenant_id: str=None,
client_id: str=None,
cdf_cluster: str="api",
cognite_project: str=None,
client_name: str=""
):
"""
Initialize authenticator class.
Args
tenant_id : The ID for the CDF tenant. Type: str
client_id : The ID for the CDF client. Type: str
cdf_cluster : The name if the CDF cluster. Type: str. Default: api
cognite_project: The name of the cognite project. Type: str
client_name : Optional name of the CDF client. Type: str. Default:
"""
self.tenant_id = tenant_id
self.client_id = client_id
self.cdf_cluster = cdf_cluster
self.cognite_project = cognite_project
self.client_name = client_name
self.base_url = f"https://{self.cdf_cluster}.cognitedata.com"
self.scopes = [f"https://{self.cdf_cluster}.cognitedata.com/.default"]
self.authority_host_uri = "https://login.microsoftonline.com"
self.authority_uri = self.authority_host_uri + "/" + self.tenant_id
self.creds = self._authenticate_device_code()
self.config = ClientConfig(
client_name=self.client_name,
project=self.cognite_project,
credentials=Token(self.creds["access_token"]),
base_url=self.base_url
)
self.client = CogniteClient(self.config)
def _authenticate_device_code(self):
app = PublicClientApplication(client_id=self.client_id, authority=self.authority_uri)
flow = app.initiate_device_flow(scopes=self.scopes)
print(flow["message"])
creds = app.acquire_token_by_device_flow(flow=flow)
return creds
if __name__ == "__main__":
aut = OIDCDeviceCode(
tenant_id=creds.tenant_id,
client_id=creds.client_id,
cdf_cluster=creds.cdf_cluster,
cognite_project=creds.cognite_project
)
print(aut.client.assets.list(limit=5).to_pandas().externalId)
Hi @arian, did using the client-id, client-secret option work for you?
Hello, I have proceed to clarify on this article which details you should use for Python SDK
Details to create the Cognite client when using Postman and Python SDK
-
Tenant ID - 48d5043c-cf70-4c49-881c-c638f5796997,
-
Client ID - 1b90ede3-271e-401b-81a0-a4d52bea3273,
-
project=publicdata,
-
CDF_CLUSTER - api
-
App name: OID-Api
You can get the client secret by using the Widget in Open Industrial Data and selecting “Others”. This is explained with more detail in
Please let me know if this resolve your concerns.
Hello @majofoal ,
This doesn’t work. Here is a trace that might be helpful (Checked with both CogniteClient from the SDK and MSAL’s ConfidentialClientApplication authentication ):
Error generating access token: invalid_resource, 400, AADSTS500011: The resource principal named https://api.cognitedata.com/user_impersonation was not found in the tenant named Cognite Hub. This can happen if the application has not been installed by the administrator of the tenant or consented to by any user in the tenant. You might have sent your authentication request to the wrong tenant.
Trace ID: abee9008-11fa-438a-a036-aa77f172e201
Correlation ID: 3555db05-b89a-4a1f-b742-aef25393c2c4
Timestamp: 2022-12-13 15:22:23Z
The fact that it doesn’t throw an “Invalid client secret” exception means that the provided client secret is correct; moreover, it correctly identifies tenant since it finds “Cognite Hub” in AAD.
As of the scope, I tested both impersonation and read modes. None worked.
Here is a minimal code to reproduce it:
# Using Cognite-SDK
cognite_base_url = 'https://api.cognitedata.com'
cognite_tenant_id = "48d5043c-cf70-4c49-881c-c638f5796997"
python_client_id = "1b90ede3-271e-401b-81a0-a4d52bea3273"
client_secret= <Client id from OID secret generator's "other" application>
creds = OAuthClientCredentials(token_url=f"https://login.microsoftonline.com/{cognite_tenant_id}/oauth2/v2.0/token",
client_id=python_client_id,
client_secret=client_secret,
scopes= f"{cognite_base_url}/user_impersonation/.default"])
cnf = ClientConfig(client_name="OID-Api", base_url=cognite_base_url, project="publicdata", credentials=creds)
c = CogniteClient(cnf)
c.login.status()
# Directly using MSAL
app = ConfidentialClientApplication(
python_client_id,
authority=f"https://login.microsoftonline.com/{cognite_tenant_id}/",
client_credential=client_secret)
scope = app.get_authorization_request_url(tcognite_base_url]).split('scope=')e1] + '/.default'
auth= app.acquire_token_for_client(pscope])
I don’t know if client_name=”OID-Api” is correct or not but it probably doesn’t matter since it is apparently not used by the SDK’s LoginAPI.
I have made a couple of convenience classes based on the OIDC examples given here. Perhaps it can be useful to you.
import atexit
from pathlib import Path
from cognite.client import CogniteClient, ClientConfig
from cognite.client.credentials import Token
from msal import PublicClientApplication, SerializableTokenCache
from publicdata_credentials import credentials as creds
class OIDCInteractiveRefresh:
"""Convenience class for authentication toward CDF with Open ID Connect interactive refresh.
Based on the following exmaple: https://github.com/cognitedata/python-oidc-authentication"""
def __init__(
self,
tenant_id: str=None,
client_id: str=None,
cdf_cluster: str="api",
cognite_project: str=None,
client_name: str=""
):
"""
Initialize authenticator class.
Args
tenant_id : The ID for the CDF tenant. Type: str
client_id : The ID for the CDF client. Type: str
cdf_cluster : The name if the CDF cluster. Type: str. Default: api
cognite_project: The name of the cognite project. Type: str
client_name : Optional name of the CDF client. Type: str. Default:
"""
self.tenant_id = tenant_id
self.client_id = client_id
self.cdf_cluster = cdf_cluster
self.cognite_project = cognite_project
self.client_name = client_name
self.cache_filename = Path("cache.bin")
self.base_url = f"https://{self.cdf_cluster}.cognitedata.com"
self.scopes = f"https://{self.cdf_cluster}.cognitedata.com/.default"]
self.authority_host_uri = "https://login.microsoftonline.com"
self.authority_uri = self.authority_host_uri + "/" + self.tenant_id
self.port = 53000
self.app = PublicClientApplication(
client_id=self.client_id,
authority=self.authority_uri,
token_cache=self._create_cache()
)
self.token = self._get_token()
self.config = ClientConfig(
client_name=self.client_name,
project=self.cognite_project,
credentials=Token(self.token),
base_url=self.base_url
)
self.client = CogniteClient(self.config)
def _create_cache(self):
cache = SerializableTokenCache()
if self.cache_filename.exists():
cache.deserialize(self.cache_filename.open().read())
atexit.register(
lambda:
self.cache_filename.open("w").write(cache.serialize()) if cache.has_state_changed else None
)
return cache
def _authenticate_azure(self, app):
accounts = app.get_accounts()
if accounts:
creds = app.acquire_token_silent(self.scopes, account=accountss0])
else:
creds = app.acquire_token_interactive(scopes=self.scopes, port=self.port)
return creds
def _get_token(self):
return self._authenticate_azure(self.app))"access_token"]
if __name__ == "__main__":
aut = OIDCInteractiveRefresh(
tenant_id=creds.tenant_id,
client_id=creds.client_id,
cdf_cluster=creds.cdf_cluster,
cognite_project=creds.cognite_project
)
print(aut.client.assets.list(limit=5).to_pandas().externalId)
And the contents of `publicdata_credentials.py`:
from types import SimpleNamespace
credentials = SimpleNamespace(**{
"tenant_id": "48d5043c-cf70-4c49-881c-c638f5796997",
"client_id": "1b90ede3-271e-401b-81a0-a4d52bea3273",
"cdf_cluster": "api",
"cognite_project": "publicdata"
})
Example usage of the class for the public data project is given in the “if __name__ == …….” statement.
@Anders Brakestad Thanks. This is a useful script to exploit token caching if you could authenticate interactively the first time (to get the account), and from then on to acquire token for that account with the specified scope. I can’t use any interactive login.
Also, as I mentioned earlier, using device code is not an option for me.
When I use this an interactive log in window pops up - is this what you refer to? Then it caches after.
When I use this an interactive log in window pops up - is this what you refer to? Then it caches after.
Yes exactly . My script is deployed remotely so I can’t do an interactive login at all. Unless I mimic a browser on a headless server
Looking at your scopes parameter it’s a bit wrong, it should only contain .default and not user_impersonation:
scopes=ef"{cognite_base_url}/.default"]