Skip to main content

When I run the following code in Jupyter notebook in CDF, I dont get any problem and it runs fine

import pandas as pd
root_asset = 'LPM_YT_MODEL'

def handle(client, data=None, secrets=None, function_call_info=None):

    start=pd.Timestamp(data["start_date"])
    end=pd.Timestamp(data["end_date"])

    ts_names_list = client.time_series.list(limit=None,asset_subtree_ids=[client.assets.list(name=root_asset)[0].id])

    time_series_data_extids = ts_names_list.as_external_ids()

    data_points = {ts: client.time_series.data.retrieve_dataframe(external_id=ts,start=start,end=end,) for ts in time_series_data_extids if not client.time_series.data.retrieve_dataframe(external_id=ts,start=start,end=end,).empty}

 

 

but when this is executed as cognite function, it is throwing following error

 

Traceback (most recent call last):
File "/home/site/wwwroot/function/_cognite_function_entry_point.py", line 455, in run_handle
result = handle(*function_argument_values)
File "/home/site/wwwroot/function/handler.py", line 12, in handle
time_series_data_extids = ts_names_list.as_external_ids()
File "/home/site/wwwroot/.python_packages/lib/site-packages/cognite/client/data_classes/_base.py", line 214, in __getattribute__
attr = super().__getattribute__(item)
AttributeError: 'TimeSeriesList' object has no attribute 'as_external_ids'

@eashwar11, can you please check which version each of them are running on? I am pretty sure the “as_external_ids” was a recent addition (to v6.15). You can check this with client.version. The solution is most likely to upgrade your requirements file in your Cognite function to the latest version.

Thanks @HaydenH . Updated the requirements with the version = 6.15.0 and the function ran till the end. 

 

Now I have a unique error. At the end of the function, I have a statement and this ran fine in a jupyter notebook.

df_final.to_csv("Extracted_data.csv")

This is the error I am getting. Please help. How Do i store this csv in CDF through this function.

I need to store this final dataframe into a csv.

 

Traceback (most recent call last):
File "/home/site/wwwroot/function/_cognite_function_entry_point.py", line 455, in run_handle
result = handle(*function_argument_values)
File "/home/site/wwwroot/function/handler.py", line 57, in handle
df_charts.to_csv('Extract_Charts.csv')
File "/home/site/wwwroot/.python_packages/lib/site-packages/pandas/core/generic.py", line 3772, in to_csv
return DataFrameRenderer(formatter).to_csv(
File "/home/site/wwwroot/.python_packages/lib/site-packages/pandas/io/formats/format.py", line 1186, in to_csv
csv_formatter.save()
File "/home/site/wwwroot/.python_packages/lib/site-packages/pandas/io/formats/csvs.py", line 240, in save
with get_handle(
File "/home/site/wwwroot/.python_packages/lib/site-packages/pandas/io/common.py", line 859, in get_handle
handle = open(
OSError: oErrno 30] Read-only file system: 'Extract_Charts.csv'

 

No worries @eashwar11, regarding the last error, what is in the Dataframe? Are the columns time series datapoints or is it tabular? Regardless, I would suggest you upload them as datapoints to timeseries objects or as a sequence. Then if you need the results in a csv, they can be converted, as you can use .to_pandas() and then pandas has a to_csv method. On Cognite Functions, you only have a Read-only file system, as indicated by the error.

Thanks @HaydenH . Customer is asking for a csv file only. The final dataframe is having a long list of columns (each column representing some specific series)

They would like to fetch the csv from CDF and then use it to analyse offline

As the error message eludes to, most of the file system is read-only, but you do have write access to the /tmp directory.

After checking the pandas documentation, I see that the .to_csv method also accepts a buffer, so you may do everything in memory and then use client.files.upload_bytes

Btw, this is a good resource for known issues with Functions: https://docs.cognite.com/cdf/functions/known_issues

Reply


Terms & ConditionsCookie Settings