C# SDK read raw data

Hi Adarsh,

You’re on the right path by using the ListRowsAsync method from our dotnet SDK to fetch rows from a RAW table. This method allows you to retrieve data from a specified RAW table in a given database.

The “project limit reached” error you mentioned indicates that you’ve hit a limit within your CDF project. These limits could relate to concurrency or data transfer rate, being tied to the API itself and independent of the SDK you’re using. I recommend consulting the Cognite API documentation for more information about the specific limits of the RAW service.

In a CDF project, it’s common to have multiple workflows running simultaneously, which can contribute to reaching a project-wide cap. Since not all simultaneous jobs run all the time, you might notice these “project limit reached” errors to be intermittent, while the limits are temporarily reached. If you cannot properly control and distribute load between all simultaneous jobs, I recommend adopting retry mechanisms in your app logic, especially for services under high load, like RAW.

Here is a minimal example of how to use the ListRowsAsync method from the dotnet SDK with a simple retry logic:

// initialize the variable to store the result
ItemsWithCursor<RawRow<Dictionary<string, JsonElement>>> res;

// set the retry count and max retries
int retryCount = 0;
int maxRetries = 5;
while (retryCount < maxRetries) {
  try {
    res = await client.Raw.ListRowsAsync<Dictionary<string, JsonElement>> (
      database: "pump_data",
      table: "maintenance_records",
      query: new RawRowQuery{Limit = 1000}
    );
    // if no exception is thrown, break the loop
    break;
  }
  // in this simple example, we will catch all exceptions
  // but in a real-world scenario, you should catch specific exceptions
  catch (Exception e) {
    retryCount++;
    Console.WriteLine($"Error: {e.Message}");
    // wait for 10 seconds before retrying
    await Task.Delay(TimeSpan.FromSeconds(10));
  }
}

In the example above, we are retrieving 1000 rows from the table maintenance_records in the database pump_data. The retry logic helps mitigate the “project limit reached” error by attempting the request again after a delay if an exception is caught. In this simple example, we retry up to 5 times with a fixed delay of 10 seconds between each attempt. In a production environment, you might want to consider more advanced retry strategies, such as exponential backoff.

If you need to paginate through the results, you can use the NextCursor property available in the response. Create a new RawRowQuery object with the Cursor property set to the value of NextCursor to fetch the next page of results.

Let me know if that helps you move forward on this topic.