Skip to main content

dear all 

I was attempting to perform data aggregation based on the date. I am retrieving online data into CDF, which is updated every 2-3 minutes. I am trying to aggregate the data so that the date is updated every 24 hours instead of every 2-4 minutes.
I used this code to obtain the list of columns in my data frame.

# Check the structure of the DataFrame, including column names and the first few rows
print(dp.columns)  # This will show all column names
dp.head()          # This will show the first few rows of the DataFrame 

however, I got only this after running the code 

I tried to use another code again the first column is not date column. 

 

Hi
Yes, you can aggregate based on the date!
https://cognite-sdk-python.readthedocs-hosted.com/en/latest/time_series.html#cognite.client._api.datapoints.DatapointsAPI.retrieve_dataframe

If you set the granularity parameter to “1d”, and the aggregates parameter to the kind of aggregate(s) you want, you will retrieve daily data points aggregates instead of every individual data point. There are also a few examples in the SDK documentation I linked to above.

By default, it will group the data from midnight to next midnight in the UTC time zone, but this can be changed using the timezone parameter.
Hope this helps :)

Matias

Hi Matias

Thanks for advise. I followed the instruction provided in the given link and it works properly. 

 

from datetime import datetime, timezone
from cognite.client.data_classes import DatapointsQuery
dp = client.time_series.data.retrieve_dataframe(
    external_id=_
        DatapointsQuery(external_id="XXXX:i=3568", aggregates="discrete_variance")
    ],
    granularity="1d",

    uniform_index=True) 

best regards 

khalfan 

 

Hi 

I faced another challenge. I am trying to insert the created data frame into CDF Time Series. As I understand you need to set index for date as it is shown in the code below

df.set_index(("Date"], inplace=True)

However, the data I have are online data and there is no title in the column of data 

Appreciate if you could support me on this regards 

As you are looking for inserting timeseries data. Please check if below link helps. 

 

 

Reply


Terms & ConditionsCookie Settings