Welcome toVigges Developer Community-Open, Learning,Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
603 views
in Technique[技术] by (71.8m points)

Debezium, Kafka connect: is there a way to send only payload and not schema?

I have an outbox postgresql table and debezium connector in kafka connect that creates kafka messages based on the added rows to the table.

The problem I am facing is with the message format. This is the created message value:

{
  "schema": {
    "type": "string",
    "optional": true,
    "name": "io.debezium.data.Json",
    "version": 1
  },
  "payload": "{"foo": "bar"}"
}

But (because of consumer) I need the message to contain only the payload, like this:

{
  ""foo": "bar""
}

This is part of my kafka connector configuration:

"transforms": "outbox",
"transforms.outbox.type": "io.debezium.transforms.outbox.EventRouter",
"transforms.outbox.route.topic.replacement": "${routedByValue}",
"transforms.outbox.route.by.field": "aggregate_type",
"transforms.outbox.table.field.event.payload.id": "aggregate_id",
"transforms.outbox.table.fields.additional.placement": "payload_type:header:__TypeId__"

Is there any way to achieve this without creating custom transformer?


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

It looks like you're using org.apache.kafka.connect.json.JsonConverter with schemas.enable=true for your value converter. When you do this it embeds the schema alongside the payload in the message.

If you set value.converter.schemas.enable=false you should get just the payload in your message.

Ref: Kafka Connect: Converters and Serialization Explained — JSON and Schemas


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to Vigges Developer Community for programmer and developer-Open, Learning and Share
...