Product User Community

I want to retrieve data frame based on external IDs for the past 24 hours. So, if current date = 08/15/2023, it should try to get the previous day’s datapoints for 24 hours i.e, from 08/14/2023 00:00:00 to 08/14/2023 23:59:59. Please share with the actual code on how to put the parameters.  from cognite.client import CogniteClientclient = CogniteClient()df = client.time_series.data.retrieve_dataframe(... id=12345,... start=<What to fill here>,... end=<what to fill here>) 

Facing a Critical Error while running the below sql code.SQL Error:(Attached the Raw input Excel for your reference.)Text '9/28/2022' could not be parsed at index 0 | code: 400 | X-Request-ID: 14d6d486-1958-99a3-b552-d19e5937bf82 -- Loads SAP Work Order as events. Data is already contextualized with assets using functional location. An inner join with CDF Assets is used to ensure only work orders for assets already loaded is ingested.select concat('WO_', `Order Number`) as externalId, to_timestamp('Basic Start Date','M/dd/yyyy') as startTime, to_timestamp(`Basic Finish Date`,'M/dd/yyyy') as endTime, 'Work Order' as type, array(`Asset ID`) as assetIds, 'SAP PM' as source, `Description` as description, `Order Type` as subtype, 2395436788557957 as dataSetId, map_concat( to_metadata_except(Array( -- "Order Number", "Basic Start Time", "Basic Start Date", "Basic Finish Date", "Asset ID", "Description", "Order Type"), *) -- map("Priority",

Hello, I am trying to use “Filter time series” or “List time series” endpoints. However, because of 1000 limit, I need to parallelize these calls as I would like to retrieve thousands of timeseries. The challenge I am facing is the pagination used in these endpoints which is based on id of timeseries. Is there a way to use index as pagination? 

Hi,To apply a startWith filter on a property, we can do:prop_view = client.data_modeling.views.retrieve(('myspace', 'myview'))filters.Prefix(prop_view[0].as_id().as_property_ref("myPropertyName"),"xxx")This is working fine. However, as the externalId is not a property so the code below doesn’t work:filters.Prefix(prop_view[0].as_id().as_property_ref("externalId"),"xxx")and the error message is: “Invalid prefix filter with property reference [myspace, myview/94b12855a09cf4, externalId]. View myspace:myview/94b12855a09cf4 does not have property externalId.” Could you help find out how to apply a startWith filter on externalId in python sdk? Regards,Xiaofeng

How do I log errors and activities while working with the Cognite SDK. I need to keep track of what the code is doing, diagnose issues, and improve the overall reliability of the application. If I am designing a function, how to handle all these logging in my code to better log errors and validations within the application. Please advise. Also share practical code samples from SDK   

I have implemented block-updates to work with CDF Files. This by itself works quite well, as documented here.However, for very large files there are some problems. After some time, typically around 4 minutes, one of the block updates fail, with a simple `ReadError`. There’s no response from the API.Some logs from our application:DEBUG:odp.odcat.storage.cdffs.async_spec:Uploaded 133120 bytes to https://westeurope-1.cognitedata.com/api/v1/files/storage/cognite/6723545981847516/7406155183547467/wod_gld_2013.nc?sig=REDACTED&rsct=application%2Foctet-stream&sp=cw&sr=bDEBUG:odp.odcat.storage.cdffs.async_spec:Total bytes uploaded: 2660989807DEBUG:odp.odcat.storage.cdffs.async_spec:Data: len=5376000, MD5=6a0c0db1b66dac18ee28e8b082830667DEBUG:odp.odcat.storage.cdffs.async_spec:Buffer cursors: start=0, end=5242880, final=FalseDEBUG:httpx._client:HTTP Request: PUT https://westeurope-1.cognitedata.com/api/v1/files/storage/cognite/6723545981847516/7406155183547467/wod_gld_2013.nc?sig=RED

We have some requirements to use DB in built extractors and how can we setup Extraction pipeline configuration. 

I am looking for a way to update the extraction.id-prefix field inside the OPC-UA extractor config file after partially starting data extraction. Basically, I need to be able to store data in CDF raw with External ID prefix set to a particular tag value coming from the OPC server. So I would need to read the timeseries value of that particular tag from the OPC server and set this value as id-prefix in the extractor config file before pushing subsequent data to CDF raw.Since the pre-built OPC-UA extractor works well for our use case, I am looking for ways to use it as is without having to implement a custom extractor. What would be the best way to implement the functionality mentioned above using the OPC-UA extractor?(Attn. @Jatin Sablok )

Hi. I’m using the Cognite Data Source for Grafana. For numerical time series I can retrieve properties like {{name}}, {{description}}, etc. But for some reason I cannot for string time series. The only difference between the two panels below is the External Id of the time series it points to: one is a string time series, the other numerical. Both time series have a name specified in CDF. 

I have some views with simple array properties (arrays of primitives) and others with arrays of relationships to views (edges).Location viewI can filter the simple arrays with the “containsAll” or “containsAny” methods.But there seems to not be a filter avaiable for the edges in the Graphql API. How can I filter by these properties via the GraphQL API?

Hi I have a final computed dataframe that looks like this. sample set alone listed here (15 rows) There are 570 records. I need to create timeseries for sumprod and derivedYields columns as per the value in Yieldcode. I have written the code and it is running and timeseries is getting created. But it is running slow. How to make it get done faster?Any best method? Please advise     sumprod Yieldcode Rank Group derivedYields MBBLD Vol LP-Class 1 532 0.009636 VBALFG1 1 VBAL 0.009636 3.524946 FG1 2 533 0 VBALLP1 2 VBAL 0 0 LP1 3 534 0.210269 VBALFN1 3 VBAL 0.218807 80.0382 FN1 4 535 0.037774 VBALLW1 4 VBAL 0 0 LW1 5 536 0.048263 VBALSK1 5 VBAL 0.094867 34.70158 SK1 6 537 0.017368 VBALHK1 6 VBAL 0 0 HK1 7 538 0.128149 VBALSR1 7 VBAL 0.142624 52.17094 SR1 8 539 0.014722 VBALDS1 8 VBAL 0 0 DS1 9 540 0.034342 VBALAG1 9 VBAL 0.0345

I have been working on this for some time now. It was all running fine till now but suddenly it is failing now, and it is giving an error for this function. Is there something that has been changed in the platform/library?DB name and table name are all accurate and was running fine till today.  lp_df_input = client.raw.rows.retrieve_dataframe(db_name="Eashwar_MOTDB-db",table_name="lp_input",limit=None,columns=None)Error: ---------------------------------------------------------------------------MissingSchema Traceback (most recent call last)Cell In[30], line 1----> 1 lp_df_input = client.raw.rows.retrieve_dataframe(db_name="Eashwar_MOTDB-db",table_name="lp_input",limit=None,columns=None)File /lib/python3.11/site-packages/cognite/client/_api/raw.py:613, in RawRowsAPI.retrieve_dataframe(self, db_name, table_name, min_last_updated_time, max_last_updated_time, columns, limit) 589 """`Retrieve rows in a table as a pandas dataframe. <https://developer.cog

Hi,I need to connect to CDF with Power BI and when trying to load the table, I get this error:It only loads a limited rows of data (probably those rows have no errors) and I cannot change the format of the fields in PBI.I already whent thru the documentation , but the solution provided there does not work in this case (there is no option to change the data format).For other datasets (other data sources), where Power BI encounters errors, the data is still loaded (with errors), but in this case, the data is not loaded and the import is stopped by the error. This is also keeping for saving the report or using the data source.The behavior needed is that PowerBI is loading the entire dataset with errors and to change the field format before importing the data.Thank you!

I’m facing the above issue while running the demo cognite custom extractor

I am looking forward to a solution for reading a csv files that has 100 columns and 200 rows. It is stored in CDF as ‘Files’. How do I read the file and then use pandas to convert the file as a dataframe?diet_f= (client.files.list(name='diet_daily.csv'))[0]diet_daily_transformed = pd.read_csv(diet_f.name)diet_daily_transformedThis isnt working and throwing an error. Please advise the best means to perform this task. 

I have a helper function that returns a CogniteClient via interactive OAuth authentication. This worked literally yesterday, but gives me an error today.This is my function definition import osfrom cognite.client import CogniteClient, ClientConfigfrom cognite.client.credentials import OAuthInteractivefrom typing import Uniondef authenticate_to_cognite_oauth( project: str="akerbp", client_name: str="Employee", client_id: Union[str, type(None)]=None, tenant_id: Union[str, type(None)]=None, base_url: str="https://api.cognitedata.com" ) -> CogniteClient: """Authenticate to Cognite via OAuth interactive login. Return cognite.client.CogniteClient.""" if client_id is None: client_id = os.getenv("CDF_CLIENT_ID") if tenant_id is None: tenant_id = os.getenv("CDF_TENANT_ID") authority_url = f"https://login.microsoftonline.com/{tenant_id}" scopes = [f"{base_url}/.default"] creds = OAuthInteractive( authority_url=authority_u

I want to create a cognite function that will do lot of computations and finally attach the derived output to a timeseries. Is there some production-ready sample code snippet that can be shared to see how to attach a derived value from cognite function to a timseries?

Hi Team, We are working on a POC , where we need to create unit test cases where we are facing different issues .For reference we are using code from Cognite HubBelow attaching our code and output expectations .Please help with the approach to resolve this issue ts_retrieved = [    TimeSeries(    id= 3973245479016710,    external_id= "Brazil:3W:Well-Bore-19B_target_throughput",    name= "Brazil:3W:Well-Bore-19B_target_throughput",    is_string= False,    metadata= {},    unit= "Barrel",    asset_id= 3332326574821917,    is_step= False,    description= "target volume for a oil well bore",    security_categories= [],    data_set_id= 8739280581150161,    created_time= "2023-05-02 09:37:04.944000",    last_updated_time= "2023-05-02 09:37:04.944000")]@pytest.fixturedef cognite_client_mock():    with monkeypatch_cognite_client() as client:          client.assets.retrieve().time_series().get().return_value = ts_retrieved        return client   Method code          ts_data = c.assets.retrieve(

Please share Cognite deployment architecture diagram. How storage is maintained on public cloud in various regions?

Getting OSError: [Errno 22] Invalid argument while I am running a query while other queries are running.And also this query is running in SQL server only giving error of invalid argument while ingesting timeseries data in extractor. 

scenarios we are working on, To show PI&D diagram with annotations, which help to select the asset and create a threat for that asset. Or draw a rectangle over the PI&D diagram to select multiple assets. For the above two scenarios we are referring APIs in,https://docs.cognite.com/api/v1/#tag/Engineering-diagrams Quickstart — cognite-sdk 5.3.1 documentation (readthedocs-hosted.com) We are successful in getting the PI&D as image by following the below steps,1. List Fileshttps://api.cognitedata.com/api/v1/projects/{project}/files2. Convert a diagram to image formathttps://api.cognitedata.com/api/v1/projects/{project}/context/diagram/convert/3. Get the results for converting an engineering diagram to an imagehttps://api.cognitedata.com/api/v1/projects/{project}/context/diagram/convert/{jobId} Could you please guide with APIs / JavaScript SDK methods we need to work on for the above mentioned two scenarios. 

I have some code working great via a Notebook, but I want to move it to a server where I can run it as daemon. The authentication is failing when I run it on the server.I suspect the problem is with the Scopes - User impersonation might work in the browser where I’m already logged in to the right project. What scope should I use instead when I’m calling directly from Python?Also tried myscope = '[f"https://westeurope-1.cognitedata.com/.default"]'as suggested in the documentationoauthlib.oauth2.rfc6749.errors.AccessDeniedError: (access_denied) Unauthorizedfrom cognite.client import CogniteClient, ClientConfigfrom cognite.client.credentials import OAuthClientCredentialsfrom cognite.client.data_classes import TimeSerieshost = "https://westeurope-1.cognitedata.com"myproject = "rok-buandcollaborators-53"clientid = "realStuffWasHere"secret = "secretsecretsecretsecretsecretsecret"#myscope = "https://westeurope-1.cognitedata.com/.default"myscope='["IDENTITY", "user_impersonation"]'token="https

I created a set of timeseries via the API. The Python examples do not show how to include an externalId nor do they say one is really needed. Looks like they need one, otherwise they don’t work in Charts. That’s all fine. My problem now is to add the externalId to existing timeseries.I think the right way to do this is with a transform, but I can’t get it right. I can get it to preview correctly, but it fails when I run it.My transform is:Select name as name, concat(name,"123") as externalIdfrom `_cdf`.`timeseries` where dataSetId = 8883655420989208;The error is Request with id 2463250f-1635-9a05-aa9a-709366e80349 to https://westeurope-1.cognitedata.com/api/v1/projects/rok-buandcollaborators-53/timeseries/update failed with status 400: Time series not found. Missing externalIds: [Tank-401-01-TT123].Looks like it’s trying to query for the timeseries with that externaId when I’m trying to set the externalId. How do I get it to set the value?

How to fetch all the timeseries linked to an asset? I have a root asset that has 16 child assets. Each of those child has 18 timeseries objects. How do I write a code so that I can loop through this setup and read each of those timeseries objects and read the datapoint for a particular day and gather them. root = client.assets.list(name='liquid_asset')children=client.assets.list(parent_ids=[root[0].id])for i in children: print(i.time_series)Result is this: <bound method Asset.time_series of <cognite.client.data_classes.assets.Asset object at 0x3468500>>I could not iterate through the i.timeseries and read the datapoint from a very specific timeseries object named as 'PVOL'. Please help. 

Hello, I am trying to create a row instance in my data model that references multiple other types, via the REST API.I can do that fine with single relatioships, but when I get the ErrorCannot ingest into a connection property: supportingCast in view 'Film:Actor/xxxxxxxxxxx'.when I try an array of types like this:{ "externalId": "Oppenheimer", "space": "imdb", "lead": { "externalId": "CillianMurphy", "space": "imdb" }, "supportingCast": [ { "externalId": "EmilyBlunt", "space": "imdb" }, { "externalId": "RobertDowneyJr", "space": "imdb" } ]}The supportingCast property is the problem is this case.