Product User Community

HiFor a use case I am working on I need to retrieve information about the raw tables in our environment. What I need to do is that i need to retrieve the name of each of the  columns and the datatypes of the columns for every single raw table we have in our environment. I have looked through the python SDK docs, but have not been able to find out how to do this. I noticed that it is possible to do a preview of a transformation through the SDK and get a schema out this way, however this feels a bit too “hacky” and is also very time consuming and inefficient. Does anyone have any idea of how I can get what I need?ThanksSebastian

Here are some issues we are facing with TimeSeriesUploadQueue after migrating to lastest version :Unable to upload rows, though it says it has been uploaded in latest versionIt says uploader uploaded x datapoints even though it doesnt (checked in ui and through api) , and uploaded rows is None even though there are datapoints to upload. Same code works fine in version 6.This is our post upload function: Wrong date range is uploaded in version 7.  (happens on a random basis)The target timeseries didnt exist before we run the extractor. The date range we were trying to upload in the first batch is October 25, 2022 11:38:38.553 PM - December 27, 2022 11:03:47.055 AM and in the second batch is December 27, 2022 7:03:47.055 PM to  January 1, 2023 11:03:47.055 AM( all GMT times)Both batch's upload failed according to sdk (error during callback, rows is NoneType), but somehow 50 datapoints in a completely different date range is written as observed through cognite ui and through sdkWe also fa

(Please forgive the erroneous posting of this under Academy Discussions)Doing the ‘hands-on’ for Data Engineer Basics - “Learn to Use the Cognite Python SDK “ Course."I have done the following:Make a dataframe from reading from a csv file: One of the columns in the csv file is ‘region’. The 5 different values in the region column are  When the above code is run, the  following appears on the screen: Q1. Does this mean pandas has interpreted the region “None” as “nan”? If so that would explain why the following code throws “ValueError: Out of range float values are not JSON compliant. Make sure your data does not contain NaN(s) or +/- Inf!”  I would value your guidance as I have not worked with pandas() much at all.Thank you,Doug  pandas  

Hi,We are using graphql query to get data points in CDF function. As I understand we could retrieve 1,00000 data points at a time if we provide the limit. Here is the sample query:query MyQuery {  listProductionData(    filter: {externalId: {in: ["wellbore1"]}}  ) {    items {      space      data {        Oil {          1_day {            dataPoints(end: 2240591400000, limit: 100000, start: 1672338600000) {              value              timestamp            }          }        }      }    }  }}I tried above query and I can get all the daily datapoints present in the range from  2023-01-01 to 2040-12-31.  But when I tried below query : query MyQuery {  listProductionData(    filter: {externalId: {in: ["wellbore1"]}}  ) {    items {      space      data {        Oil {          1_day {            getDataPoints(start: 1672338600000, end: 2240591400000) {              items {                timestamp                value              }            }          }        }      }    }  }}we c

Hi all,I'm trying to increment my view version.To do this, I'm also incrementing my data model version as well.My original is```type Test @view(space: "test_space", version: "1_0"){    name: String!,    description: String,}```and I try to go to```type Test @view(space: "test_space", version: "1_1"){    name: String!,    description: String,    test_write: String}```I've also tried ```type Test @view(space: "test_space", version: "1_2"){    name: String!,    description: String,    test_write: String,    required_test_write: String!}```I get this error: {  "title": "Error: could not update data model",  "message": "An error has occured. Data model was not published.",  "extra": {    "type": "div",    "key": "errors",    "ref": null,    "props": {      "style": {        "display": "block",        "overflowX": "hidden",        "overflowY": "auto",        "maxHeight": "150px"      },      "children": {        "type": "ul",        "key": null,        "ref": null,        "props": {         

Hey everyone ! Welcome back to our data modeling adventure in CDF! In my first post, I tried to explain my understanding of the core concepts - spaces, containers, views, and the elusive knowledge graph. We learned how data models act as the foundation for unlocking the power of Cognite Data Fusion for your industrial data.This time, let's delve deeper into the world of views and containers - the essential ingredients that bring your data models to life! Beyond the Fundamentals: Unveiling Views and ContainersThink of a data model as a recipe. It doesn't contain the actual ingredients (data) itself, but it tells you exactly what pre-made dishes (views) and fresh ingredients (containers) to use. Views: The Pre-Assembled Data: Imagine views as pre-prepared dishes – Views are essentially pre-defined data selections curated from specific containers. These containers hold the raw data (think sensor readings or equipment properties). A single view, akin to a pre-made sauce, can be referenced

I am facing this error in the data science course on “creating cognite functions” with cognite SDK. In previous courses I had fixed this error by replacing the “datapoints” keyword with “time_series”. However, I would like to know if I am probably not using the write packages; OR not know if the commands are deprecated and have new function names in newer versions. Kindly let me know - I am trying to finish these courses before my local boot camp next week.Thanks,Lavanya

Hello,I’m encountering an issue with accessing a dataset on Cognite Jupyter Notebooks. According to this link: Jupyter Notebook in Cognite Data Fusion | Cognite Documentation, subfolders will appear empty if I don’t have access to the dataset. However, I can see the dataset through the data model corresponding to that dataset, but I still can't see any contents inside the subfolders.Do I need anotther type of access or is it another issue? Any help would be much appreciated.Thanks!

Hey guys. I’m trying to create a function with some different constants. Do you know how can I change the name of the constants? 

Hi,I have some questions on Cognite. Would you please help?1. Files1.1 To delete Cognite files, after deleting the files do we need to delete the documents also? 1.2 After we create the files, the documents are created by Cognite automatically, but it seems there are some random delay to create the document, why?1.3 what is the concept of document in Cognite? Why is it created? Any documents on that?2. sequences2.1 To delete Cognite sequences, after deleting the sequences do we need to delete the datapoints also? 3. timeseries3.1 To delete Cognite timeseries, after deleting the timeseries do we need to delete the datapoints also?4. delete order for containers, views 4.1 to delete all FDM data in a space, the order of deletion should be edges, nodes, containers, view versions, data models, space. And the containers should be deleted first then delete the view versions, right?Thank you!Alice

I’m working on a prototype for a flexible data model to store time series data in a way that is easy to catalogue, query and filter. Using Pygen both to populate and use the model seems convenient.At its current iteration, I’ve only applied direct relations and (undocumented?) @reverseDirectRelations in the GraphQL schema. I expected to be able do something similar to client.windmill(windfarm="Hornsea 1").blades(limit=-1).sensor_positions(limit=-1).query()as found in the Pygen documentation, but it does not work (my client.windmill analouge has no methods corresponding to its relations). Do I have to use edges instead of relations to query easily and declaratively with Pygen?

This document outlines a concept that CDF has been under development for the last 2 years. As mentioned during the disclaimer above, please use this Cognite Hub group and share any feedback you have around flexible data modeling.This document is reflecting some very early thinking from the App Dev Journey team, and is a mental model that will likely change overtime with your feedback!What is a Data Model​A data model enables users to customize the shape, structure their expectation of data. It plays a crucial part in building solutions (like data science models, mobile and web apps). But it is also is the core of an ontology, knowledge graphs, or industry standard.There are some crucial reasons why data models are effective for the industrial space. Data modeling enables explicit language, flexible customization, governed iteration, and enhanced accessibility towards data. Let's dive further into each of these qualities of data modeling. Data Model is Explicit​A data model needs to be

Dear community,We have been working with Python SDK developing functions and everything works fine, as expected.Now we have an IP development that has C# as base language. We need to call some of the Cognite Functions utilizing the .net SDK (https://github.com/cognitedata/cognite-sdk-dotnet)  but we are having troubles to call the functions trough .net.Has anyone been able to utilize Cognite Functions trough the .net SDK?Thank you all.Here is a snippet of the code: 

I am trying to create a Data Model in the UI. I can click ‘create data model’, name it, and land on the page with the editor and the pre commented pump/facility/documenation example. However I am not able to edit the graphql as anytime I try to edit in the editor I am told that I ‘cannot edit in read-only editor. Where is the toggle to disable the read-only editor?

After how much time retry of function get triggered on failing?Is there a way we can set the time to retrigger the function after function fails. It will be helpful to deal with rate limit in functions if any.

Hello, how can I delete all the time series of a dataset with python sdk?

Hello,I’m trying to consume data from FDM in Power BI. But I’m getting error (408) Request Timeout. I’m already filtering Power BI for only a few days of data, which brings an amount of data comparable to the amounts I have already consumed in Power BI. How can I solve it?

Hello,I am trying to follow along the first chapter in Data Engineer Basics - Integrate. I am trying to follow the first workbook : Authentication. When I try to install poetry, I get an error below.Can anyone let me know how I may troubleshoot? I am on mac, python 12 and pipx (instead of pip)  at ~/Library/Application Support/pypoetry/venv/lib/python3.12/site-packages/poetry/installation/chef.py:164 in _prepare      160│       161│                 error = ChefBuildError("\n\n".join(message_parts))      162│       163│             if error is not None:    → 164│                 raise error from None      165│       166│             return path      167│       168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:Note: This error originates from the build backend, and is likely not a problem with poetry but with pyzmq (25.0.2) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "pyzmq (==25.0.2)"'.Th

Hi everyone! 👋We are thrilled to announce that the next major release of Cognite Data Fusion is just around the corner, launching on June 4th, 2024. This update is packed with exciting new features across our Industrial Tools and Data Operations capabilities.Head over to Product Updates for a sneak peek of what's coming! Tip: hit the subscribe button on Product Ideas, and you'll be notified instantly when our product leaders share product updates.We look forward to your feedback and appreciate your continued contributions!

What access capabilities do I need to run transformations as “Current User”. I have a user who don’t see “Run as current user” as in screen shot. Next screen shot is mine, and I can see it probably because I am added as admin 

Hi, Is it possible to use multiple clients for the same Cognite Function? I know the handle function only takes one client as argument, but is it possible to initialize another client inside the function? I want my Cognite function to read from prod environment, but write to dev environment, which necessitates two clients. Thanks.

Semantic search is a new service under the existing Documents API in Cognite Data Fusion. For details on what semantic search is and an introduction to the service, please see attached presentation below.The service lets you search for relevant parts (passages) of up to 100 documents by using advanced filters and semantic search queries. Semantic search works by comparing the semantic meaning of the search query to the semantic meaning of the document passages. The document passages are automatically derived and indexed upon file uploads (PDF only).The flow for using semantic search on set of documents is as follows: Upload the PDF documents to Cognite Data Fusion Query /documents/status to check if the document is ready  Query /documents/semantic/search to find relevant passages in the document(s)Documentation for the service is being developed. Please find documentation for the respective endpoints below:/documents/status /documents/semantic/search

Deployment of workflow , can you guide how we can automate deployment of cognite workflows using sdk 

Hey Data Workflows users!TL;DR If you’re using the workflow execution cancellation endpoint, with the python SDK or without, you will have to update your workflow execution cancellation calls before 29/05/2024. More information below.In our push towards making the Data Workflows API generally available in the June release, we aim to ensure a consistent and expected experience across the API. For this reason, we’ve decided to make a breaking change to the workflow execution cancellation endpoint. Endpoint changesIn summary, the cancellation endpoint now only allows the cancellation of a single execution at a time instead of allowing multiple executions to be cancelled in one call. The updated API documentation can be found here. The python SDK, starting from version 7.42.0, will now point to the new endpoint. An example call to the new endpoint using the Python SDK can be found here.TimelineThe new endpoint is already available for use. To assist with the transition, the old endpoint wi