Skip to main content
Answer

MQTT Extractor - Update Events

  • November 21, 2023
  • 5 replies
  • 85 views

Within custom transformations, is there a way of updating ongoing events in cases where the MQTT messages does not contain an UUID or enough information to build an event-externalId?

In the example messages below I would ideally like to create an event based off the first message, and then update it when the second message is received. Is it possible to do so without an UUID that is contained within both messages?

 

Events start message:

{
    "alarm": {
        "data": {
            "Name": "logic_cond",
            "Source": "logic",
        },
        "id": "logic",
        "message": "TOOL UNCLAMPED",
        "state": "active",
        "timestamp": "2023-11-08T14:16:41.626645Z"
    },
    "source": {
        "applicationType": "mtc",
        "clientID": "asset5",
        "ownerID": ""
    }
}


Event end message:

{
    "alarm": {
        "data": {
            "Name": "logic_cond",
            "Source": "logic",
        },
        "id": "logic",
        "message": "TOOL UNCLAMPED",
        "state": "dismissed",
        "timestamp": "2023-11-08T14:25:32.116245Z"
    },
    "source": {
        "applicationType": "mtc",
        "clientID": "asset5",
        "ownerID": ""
    }
}

 

 

Best answer by mathialo

The extractor does upserts based on external IDs, so if the external ID changes, a new event would be created.

There’s currently no way of fetching data from CDF during the transform, so fetching an event and then changing its external ID is not possible unfortunately.

5 replies

  • Practitioner
  • November 21, 2023

Without an external ID it would not be possible to uniquely identify the event that the extractor is going to update, so we need that in order to update an event.

 

As far as I see, there aren’t any info in the first message that’s missing from the second. Could you generate an external ID with the `concat` function? E.g. something like this:

concat("mqtt-prefix/", input.source.applicationType, "-", input.source.clientId)

 


  • Author
  • Seasoned
  • November 21, 2023

As far as I see, there aren’t any info in the first message that’s missing from the second. Could you generate an external ID with the `concat` function? E.g. something like this:

concat("mqtt-prefix/", input.source.applicationType, "-", input.source.clientId)

 

It would make it unique across ongoing events but sadly it wouldn't be unique across all events from this clientID, there are several messages that share identical source- and data-objects, so the only way of making it unique would be to add the timestamp that will be inaccessible by the time the end-message is received.


  • Author
  • Seasoned
  • November 22, 2023

Is it, or will it be possible to update the externalId of an event through Kuiper?


  • Practitioner
  • Answer
  • December 1, 2023

The extractor does upserts based on external IDs, so if the external ID changes, a new event would be created.

There’s currently no way of fetching data from CDF during the transform, so fetching an event and then changing its external ID is not possible unfortunately.


  • Author
  • Seasoned
  • December 1, 2023

That makes sense, thanks.