How to get started with a sandbox using Cognite replicator [Cognite Official]

If you have a sandbox that you would like to populate with data to get started with Cognite Data Fusion, you can use Open Industrial Data. We have built a Cognite replicator for that purpose.

Disclaimer: This component is Community content and not officially supported by Cognite. Bugs and changes will be fixed on a best effort basis. Feel free to open issues and pull requests, we will review them as soon as we can.

How to use the Cognite replicator?

You have detailed instructions on Github. You have information on how to run it as python package, docker container or a standalone script and authentication via user impersonation and client credentials.

How do I authenticate to my source and destination project? What permissions do I need?

You have 2 ways of authenticating to your source and your destination:

• via user impersonation. the script will be run on behalf of your user, you should therefore have the required permissions (read/write on the resources you want to write to/from + groups:list and projects:list). You can use this when you have access to a browser to authenticate.
• via client credentials: app registration on your identity provider with a secret and in a group with the required permissions (read/write on the resources you want to write to/from + groups:list and projects:list). You should use this if you don’t have access to a browser or wants to run the script automatically, for example via a service account.

You can authenticate the same way to both projects and via different methods depending on your configuration.

How do I connect to Open Industrial data as the source of my replication?

You can find more information here:

• If you want to run the replicator once and get a batch of data, you can use standard authentication for python SDK
• If you want to run the replicator on schedule, you should request a secret

How to configure the Cognite replicator?

You can run it:

1. With a configuration file as a standalone script

Create a configuration file based on the config/default.yml and update the values corresponding to your environment

If no file is specified then replicator will use config/default.yml.

You can then choose to run via python or docker

Check here for more information

2. As a python library without configuration file and interactive login

It will copy everything from source to destination and use your own credentials to run the code, you need to have the right permissions to read on the source project and write on the destination project

More info here

3. Without configuration file and with client credentials authentication

It will copy everything from source to destination and use your own credentials to run the code, you need to have the right permissions to read on the source project and write on the destination project

More info here

4. Without configuration file but having some elements of the config as variable

More info here

5. As a standalone script with configuration file

It will use the configuration file to determine what will be copied, you can see a default configuration file in the repository here

This will give you more flexibility in terms of configuring the extractor, for example, you can list the external ids of the resources you want to copy

More info here

If you have any questions, feel free to ask here on the thread.