Skip to main content

Welcome to the Time Series Subscriptions community


Glen Sykes
Seasoned Practitioner

Welcome to the Time Series Subscriptions community group for Early Adopters!

Time series subscriptions are aimed at data engineers who need to monitor changes to Time Series datapoints / write only the updated datapoints to their application storage etc.  This feature negates the need for consumers to download an entire time series again, just to get the changes.  The feature is now available in public beta (including Python SDK support), and customers are encouraged to try it out.

This group is staffed by the product team and we are available for Q&A, providing support and receiving feedback on the feature.

To get started, please take a look at the user guide, the API specification.

We look forward to supporting you!

8 replies

  • New
  • 1 reply
  • October 10, 2023

If I want to read timeseries datapoints outside CDF, is using subscription the best way to do it? I feel the subscription is mostly used for capturing datapoints changes instead of retrieving datapoints. Let me know. 

If using subscription is better, could you show me both C# examples and Python examples?

Thank you


Glen Sykes
Seasoned Practitioner
  • Author
  • Seasoned Practitioner
  • 123 replies
  • October 11, 2023

Hi @johnpiger 

Subscriptions only retrieve changes to data points from the time after the subscription was created.  So if you’re looking to get all the datapoints, and then ensure your system remains updated with the changes, then I’d recommend two steps. 

Firstly, create the subscription to your time series, and secondly, using either the list or retrieve API endpoint, get all the data points you want that were created prior to the subscription.

I cannot offer you any sample C# code, but the Python SDK has been updated to support the Subscriptions API already.  Here’s a link to the changelog https://github.com/cognitedata/cognite-sdk-python/blob/master/CHANGELOG.md#added-24

 


Hi Glen,

First, thank you for this new api!

I am writing to seek assistance regarding a recurring issue. I am facing while working with this new API. I have been encountering frequent timeout errors (cognite.client.exceptions.CogniteReadTimeout) that are hindering my ability to efficiently utilize the service. I use the same code as documentation example :
 

for batch in c.time_series.subscriptions.iterate_data("my_subscription", "Now"):
    print(f"Added {len(batch.subscription_changes.added)} timeseries")
    print(f"Removed {len(batch.subscription_changes.removed)} timeseries")
    print(f"Changed timeseries data in {len(batch.updates)} updates")
    if not batch.has_next:
        time.sleep(1)

 

Thank you for your time and support. I am looking forward to your prompt response. If there are any further details or information required from my end, please let me know, and I will provide them promptly.-

Best regards,

Oussama

 


I also wanted to know what the conditions are for this error to occur:  requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))


Glen Sykes
Seasoned Practitioner
  • Author
  • Seasoned Practitioner
  • 123 replies
  • November 27, 2023

Thanks Oussama, one of our developers will be along soon to help out!


  • Practitioner
  • 3 replies
  • November 27, 2023

Hi Oussama!

 

We will look into this issue, but we might need some more info to speed up the investigation.

Could you share some more details on your use of subscriptions?

In which CDF project is this happening?

What proportion of calls are timing out?

Is this the only subscription on your project, or do you have many?

Are you using the default timeout of 30 seconds?

How much data is typically present in each response?

 

Best regards,

Lars Hagen


Thank you Lars, I sent you a private message !


@Lars Hagen 

I work with Celanese and we are trying to work on creating Power BI dashboards for streaming Data (Timeseries).  I am following below solution for doing the same. 


I am facing same “timeout” error while creating Subscriptions. I am using below code. 
my_filter = f.Prefix(
    property=p.external_id,
    value="EVE"
)
sub = DataPointSubscriptionCreate(
    external_id="powerbi-subscription-test",
    name="PowerBI Subscription Test",
    description="Subscription for PowerBI",
    partition_count=1,
    filter=my_filter
)
client.time_series.subscriptions.create(sub)


Reply


Cookie Policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie Settings