Solved

cognite function error

  • 29 August 2023
  • 6 replies
  • 64 views
E
Rank
Badge +2

When I run the following code in Jupyter notebook in CDF, I dont get any problem and it runs fine

import pandas as pd
root_asset = 'LPM_YT_MODEL'

def handle(client, data=None, secrets=None, function_call_info=None):

    start=pd.Timestamp(data["start_date"])
    end=pd.Timestamp(data["end_date"])

    ts_names_list = client.time_series.list(limit=None,asset_subtree_ids=[client.assets.list(name=root_asset)[0].id])

    time_series_data_extids = ts_names_list.as_external_ids()

    data_points = {ts: client.time_series.data.retrieve_dataframe(external_id=ts,start=start,end=end,) for ts in time_series_data_extids if not client.time_series.data.retrieve_dataframe(external_id=ts,start=start,end=end,).empty}

 

 

but when this is executed as cognite function, it is throwing following error

 

Traceback (most recent call last):
File "/home/site/wwwroot/function/_cognite_function_entry_point.py", line 455, in run_handle
result = handle(*function_argument_values)
File "/home/site/wwwroot/function/handler.py", line 12, in handle
time_series_data_extids = ts_names_list.as_external_ids()
File "/home/site/wwwroot/.python_packages/lib/site-packages/cognite/client/data_classes/_base.py", line 214, in __getattribute__
attr = super().__getattribute__(item)
AttributeError: 'TimeSeriesList' object has no attribute 'as_external_ids'

icon

Best answer by Håkon V. Treider 29 August 2023, 21:01

View original

6 replies

H
Rank

@eashwar11, can you please check which version each of them are running on? I am pretty sure the “as_external_ids” was a recent addition (to v6.15). You can check this with client.version. The solution is most likely to upgrade your requirements file in your Cognite function to the latest version.

E
Rank
Badge +2

Thanks @HaydenH . Updated the requirements with the version = 6.15.0 and the function ran till the end. 

 

Now I have a unique error. At the end of the function, I have a statement and this ran fine in a jupyter notebook.

df_final.to_csv("Extracted_data.csv")

This is the error I am getting. Please help. How Do i store this csv in CDF through this function.

I need to store this final dataframe into a csv.

 

Traceback (most recent call last):
File "/home/site/wwwroot/function/_cognite_function_entry_point.py", line 455, in run_handle
result = handle(*function_argument_values)
File "/home/site/wwwroot/function/handler.py", line 57, in handle
df_charts.to_csv('Extract_Charts.csv')
File "/home/site/wwwroot/.python_packages/lib/site-packages/pandas/core/generic.py", line 3772, in to_csv
return DataFrameRenderer(formatter).to_csv(
File "/home/site/wwwroot/.python_packages/lib/site-packages/pandas/io/formats/format.py", line 1186, in to_csv
csv_formatter.save()
File "/home/site/wwwroot/.python_packages/lib/site-packages/pandas/io/formats/csvs.py", line 240, in save
with get_handle(
File "/home/site/wwwroot/.python_packages/lib/site-packages/pandas/io/common.py", line 859, in get_handle
handle = open(
OSError: [Errno 30] Read-only file system: 'Extract_Charts.csv'

 

H
Rank

No worries @eashwar11, regarding the last error, what is in the Dataframe? Are the columns time series datapoints or is it tabular? Regardless, I would suggest you upload them as datapoints to timeseries objects or as a sequence. Then if you need the results in a csv, they can be converted, as you can use .to_pandas() and then pandas has a to_csv method. On Cognite Functions, you only have a Read-only file system, as indicated by the error.

E
Rank
Badge +2

Thanks @HaydenH . Customer is asking for a csv file only. The final dataframe is having a long list of columns (each column representing some specific series)

They would like to fetch the csv from CDF and then use it to analyse offline

H
Rank
Userlevel 4
Badge

As the error message eludes to, most of the file system is read-only, but you do have write access to the /tmp directory.

After checking the pandas documentation, I see that the .to_csv method also accepts a buffer, so you may do everything in memory and then use client.files.upload_bytes

H
Rank
Userlevel 4
Badge

Btw, this is a good resource for known issues with Functions: https://docs.cognite.com/cdf/functions/known_issues

Reply


Terms & ConditionsCookie Settings