Skip to main content

Hi All,

I am working with hosted extractors for kafka and it works pretty well for me with transformations when we have plain json data in kafka topics.

Now I am trying to check if we can work with zlib gzip compressed data coming in topic, I have json string and messagepayload attribute of json string will be holding compressed data instead of whole message as compressed one, is it possible to write transformation for such data.

 

e.g., in kafka topics

{
    "Header": {
        "MessageId": 133367162,
        "MessageType": "DATA_REPORT",
        "Timestamp": 1741122422,
        "PayloadCompression": "Z_LIB_COMPRESSION"        
    },
    "MessagePayload": "eJyqVnJJLctMTi1WsoquVvJLzE1VslIyVNJRckksSQxJTIeIhySmQ6WCA3wVfFMTi0uLUlNgqioLQDIu/qFOPq7xYY4+oa5KOkphiTmlMLNCMnNTi0sScwuUrAzNTQwNjYxMjAwtzA11lAJLE3MySyqVrAxqY2tjawEBAAD//x5aKt0="
}

This looks like a Base64 encoded “MessagePayload”, which I guess is compressed before it’s being Base64 encoded?

This is not something we support in the extractor.

 

We do support a fully compressed message (the decompression is configurable in the setup for the extractor). But we do not have any methods for parsing the message and then unencoding and uncompressing portions of the message “on the fly”, sorry.

What is the use case that results in this formatting?

Reply


Terms & ConditionsCookie Settings