How to Operationalize Data Workflows in CDF using Streamlit UI [Contributed by SLB]

Objective

This document demonstrates how Streamlit, a lightweight Python-based web framework, can be used to create a user-friendly interface for triggering Cognite Data Fusion (CDF) workflows - including:

• Image/File Ingestion
• Ad-hoc Workflow Execution
• Metadata tagging and status handling

Such solutions allow non-technical users (e.g., engineers, operators) to initiate sophisticated data operations without writing code -increasing accessibility, automation, and operational agility.

Architecture Overview

[User] → [Streamlit UI] → [CDF SDK/API] → [Files, Workflows, Datasets, Metadata]

• Ingestion: Upload images to Cognite File storage with metadata.
• Transformation/Processing: Trigger workflows (e.g., YOLO object detection, anomaly detection).
• Visualization & Logging: Provide real-time feedback and results to users.

Streamlit UI Features

1. File Upload to Cognite

• Accepts multiple image types (jpg, png, etc.)
• Uploads them to Cognite File Storage
• Assigns metadata and associates with a predefined data_set_id
• Handles and displays upload success or failure

client.files.upload_bytes(
    name=file_name,
    content=file_bytes,
    source="streamlit",
    external_id=f"streamlit-img-{file_name}",
    mime_type=uploaded_file.type,
    metadata={"type": "image", "source": "streamlit"},
    data_set_id="YOUR_DATASET_ID"
)

2. Workflow Trigger

• Simple button to run any registered CDF Workflow
• Option to specify version or parameters
• Can be extended to support user-input parameters for dynamic execution

client.workflows.executions.run(workflow_external_id="MainWorkflow", version="3")

Example Use Case: Subsea Image Anomaly Detection

Scenario:

1. Field engineer uploads new batch of pipeline inspection images.

2. They trigger the MainWorkflow which:

• Ingests the images

• Parses and tags them

• Runs YOLO-based defect detection

• Logs results back to CDF

3. Results are visualized in downstream dashboards or reported via alerts.

Benefits

How to Extend

• Add parameter inputs to Streamlit (text boxes, dropdowns).

• Visualize execution results (success/failure, tags, bounding boxes).

• Add authentication and user-specific tracking.

• Chain multiple workflows: e.g., QC → Detection → Report Generation.

Summary

Streamlit + CDF = Lightweight Operationalization Layer

This integration pattern is ideal for:

• Image processing

• IoT data snapshots

• AI-driven diagnostics

• Interactive ETL tasks

• Citizen data scientist enablement

It empowers asset teams to convert raw data into insights instantly - closing the loop between field data and digital action.