Impact 2024: The Industrial Data and AI Conference for and by Users | Nominate Speakers Now for a Ch...
Hi @Anders Albert, I’ve added your feedback as a bug report in our internal ticketing system, against the Python SDK. Engineering will consider the impact of modifying the current behavior, and can update this topic once we know more about if/how and when we may be able to address it. Sorry about the delay in getting back to you! Hi @Thomas Sjølshagen, I have ended up implementing the change myself and a PR is submitted for the Python SDK, which I expect will be reviewed today.
This is now released with version 2.47.0 of the cognite-sdk.
If you want to use the Python-SDK, there is a method in the FilesAPI: https://cognite-sdk-python.readthedocs-hosted.com/en/latest/core_data_model.html#upload-a-file-or-directoryYou would call it for example like this to upload a P&ID, which I assume is in a PDF format on your local drive.from cognite.client import CogniteClientfrom pathlib import Pathclient = CogniteClient()data_set_id = client.data_sets.retrieve("src:p&id").idmy_PandID = Path("data/my_super_P&ID.pdf")client.files.upload( my_PandID, external_id=f"file:p&id:{my_PandID.name.replace(' ', '_')}", name=my_PandID.name, mime_type="application/pdf", data_set_id=data_set_id,)
Unfortunately, we do not have that today, but will log it as a feature request. Just to specify this even more, I guess the method above could be?CogniteClient.default_azure_credentials(project, cluster) with the client name as optional as in the other factory methods we have (set to your user name by default)
Two thingsIn the example notebook, lets use the factory method for creating the client to avoid the URLs: from cognite.client import CogniteClient# Enter CDF and credential detialscdf_cluster = "" # "api", "greenfield", etc.cdf_project = "" # CDF project name tenant_id = "" # IdP tenant IDclient_id = "" # IdP client ID client_secret = "" # IdP client secretclient = CogniteClient.default_oauth_client_credentials( cdf_project, cdf_cluster, tenant_id, client_id, client_secret) 2. In addition, lets include the code for how to add the correct capabilities through the SDK?
@Sindre Tøsse If you have a good solution for this, can you create a PR for the SDK? I will can take over if you set up the first pass on it.
Yes, you would have to update the dataset you want to archieve as follows client.data_sets.update(DataSetUpdate(external_id="my_dataset").metadata.add({"archived": "true"})) How do you delete the files? Using the SDK should delete the file and the filemetadat: https://cognite-sdk-python.readthedocs-hosted.com/en/latest/files.html#cognite.client._api.files.FilesAPI.delete Deleting data models works. However, note that you will not get an exception if the data model does not exist. It will return an empty list if it doesn't find anything, or the list of the data models that is has deleted. Could there be a typo in the data modeling ID?
Great to hear you enjoy the tool and has taken the time to write up your feedback. Will you supply patterns or guidance for structuring the custom modules. Do you have any thoughts on which level a project should group their artifacts into a module.Yes this documentation is on the roadmap. For now, we recommend that you gather resources into a data pipeline, for example, groups, transformation, datasets, data models related to ingesting data from a source into CDF.Is there a plan for added support for the development lifecycle, e.g. downloading a full solution as configuration (from existing project), download buttons on other resources than transformation (e.g. datasets).Yes, we have plans for supporting lifecycle. Currently, exactly how that implementation will be and the user interface for it is not yet decided. It would perhaps be better to let the RAW config support multiple tables (one per DB), one file per table seems a bit bloated. A “Download CLI” on the DB level would also be
Thanks Marius for all the feedback. I have logged it, and we will improve the tool+docs based on it. And sorry for not answering your questions from the last post: I cannot see a way to add multiple tables according to the docs. The API also has DBs and tables as separate actions, so that may be the reason.This is a now a task to update the docs.As for the active flag for transformation schedules, it is correct that I mean the isPaused attribute. If we change it and redeploy it doesn’t seem to affect the target state.Will look into thisAlso, is it possible to have a shared config (for all environments)? I cannot see if it is described in the docs. This will be used for values that are shared across the environments, e.g. tenant ID/URL.You could in an older alpha version, so likely the docs are outdated. We removed it as it easily leads to increased complexity compared to having configs separate for each environment.Lastly, is it possible to have a “init new project” with just a bare mi
Hi Amir, thanks for your report and suggestions. Name is not a required field for data models, if you check the API specs, you can see that only space, external_id and version are required when creating a new data model. I cannot recreate what you are reporting here. If you have a data model, set the description and name these will be updated, including if you set them to None.I will thus close the issue above.
I have looked into the issue of isPaused:As for the active flag for transformation schedules, it is correct that I mean the isPaused attribute. If we change it and redeploy it doesn’t seem to affect the target state.I cannot recreate it. I have deployed and flipped it and gotten the desired result in Fusion. Could it be that you changed it in the source map, and did not run cdf-tk build? A mistake I have done many times, which you will be warned about in future versions of the tookit. Another mistake I did when looking into it was interpreting isPaused: true to mean the transformation schedule should run, which is wrong, running the transformation is isPaused: false.
Sorry, I missed that. You are correct the docs are outdated. Will add a task to follow up on that.
Yes, that is correct. As you see from the documentation, Pygen supports querying Python based as you are referring above and using GraphQL. The Python based querying has some limitations were this is one of them. I am interested, can you share a bit more about what you are trying to achieve? What type of syntax would you expect to work for the query you are writing?
So with the query `client.profile(phase="Gas").well(limit=-1).hub(name="Hub1").query()` you expect to get the profiles of the hub Hub1? That is the reverse of how it is working today. That query, if you had edges, would give you all profiles of phase gas, then all wells connected to these, and then all the hubs for the wells connected to all the wells. I have been looking for an example like this, so I very much appreciate you sharing this. Will look into how it can be implemented. To your last question, yes, if you switch to edges that would affect the population and the ‘free’ reverse direct relations. I think it is better to add support for querying along direct relations for pygen. I have registered both as feature request: Support querying along direct relations, and, querying in reverse.
Yes, it would be something like that. Currently, the highest priority of pygen is to robustify it against bugs and not new features. So it is currently hard to give any concrete estimate on how long it is going to take.
Hard to say without full context, but I would maybe use GraphQL queries.
Hi Kristian, we will look into this shortly.
Already have an account? Login
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.
Sorry, our virus scanner detected that this file isn't safe to download.