Exciting Updates: Enhanced Data Integration in Cognite Data Fusion (CDF)

Greetings, everyone!

We're thrilled to share some exciting updates regarding our ongoing efforts to enhance data integration within CDF (Cognite Data Fusion). These improvements are aimed at streamlining your experience and empowering you with more capabilities. Let's dive right in!

New Features for the File Extractor: 

1. OpenText Documentum REST Services Support:

We've expanded the capabilities of the Cognite File Extractor to now seamlessly integrate with OpenText Documentum REST services, so that you can connect to multiple sources by only configuring one extractor. Check out the documentation for configuration details here.

2. Advanced Filtering:

Enjoy enhanced client-side filtering options based on metadata, allowing for precise filtration and complex rules tailored to your needs.

3. Improved SharePoint Integration:

Say hello to improved SharePoint support! We now offer support for multiple SharePoint sites, dynamic site discovery, and the flexibility to configure sites via URL. Plus, we've added the capability to detect deleted files, ensuring your data remains up-to-date.

4. Enhanced File Handling:

Say goodbye to size limitations! We now support files larger than 5 GB, facilitating seamless handling of larger files.

Updates and Changes:

1. Continuous Background Uploads:

File uploads now occur continuously in the background and in parallel, resulting in significantly faster speeds. Additionally, file traversal on sources has been optimized for a smoother experience.

2. Revised Configuration Schema:

We've refined the configuration schema for FTP and SFTP, aligning it with other file providers for consistency. Refer to our documentation for detailed parameter names.

3. Improved Retries and Handling:

Experience more robust retries and improved handling of interrupt signals, minimizing disruptions during operations.

4. External IDs Enhancement:

We've revamped the external IDs for local and Samba files, now based on file paths instead of underlying inodes. This ensures a seamless transition if you're migrating from a previous version, meaning that your files will be given a new external ID if you are migrating from version 1.

Deprecation of Old Documentum Extractor:

By adding support for D2 in the Cognite File Extractor, we are discontinuing our Documentum extractor. We encourage users to transition to the more robust File Extractor. While the Documentum extractor will continue to function as usual, please note that bug fixes and support will no longer be provided.

Introducing Public Beta Service for Azure EventHub and Kafka:

We're excited to introduce a public Beta service to connect with Kafka. Seamlessly access Kafka messages with minimal configuration, and enjoy full hosted integration service within CDF.

Thank you for joining us on this journey of innovation and excellence! Stay tuned for more updates and improvements on the horizon.

Warm regards,

Elka Sierra