Flink CDC2Kafka 总结

前些天发现了一个巨牛的人工智能学习网站，通俗易懂，风趣幽默，忍不住给大家分享一下。点击跳转到网站：https://www.captainai.net/dongkelun

前言

本文主要记录 CDC2Kafka 使用过程中的一些细节配置，以 mysql-cdc 为例。

背景需求

之前 CDC 的主要使用场景并没有写到 Kafka ，而是直接 Sink 到支持更新的目标端，比如 关系型数据库、Hudi、HBase，现在有了写到 Kafka 的需求，主要原因是有的项目的厂商不会使用数据湖表格式(Hudi、Iceberg、Delta Lake)，在使用 Hive 时不能支持ACID ，那么只能通过覆盖分区的方式实现更新删除，而覆盖更新对比小表来说可以通过定时全量抽取实现，对于大表每次都全量抽取效率太慢，所以最好可以通过增量变更日志+历史Hive表数据合并来实现，对于这种需求就得通过 CDC 格式来实现，CDC 格式有 debezium-json、canal-json、maxwell-json、ogg-json、changelog-json。 关系型数据库、Hudi、HBase等是不支持CDC格式的，常用的比较熟悉的组件就是Kafka，另外后面发现直接写 HDFS 文件也是可以的，本文先总结 CDC2Kafka+debezium-json 中的一些具体需求配置。

debezium-json

格式数据示例：

1
2
3
4
5
6
{"before":{"id":3,"name":"hudi3","price":3.33,"ts":2000,"dt":"20230331"},"after":null,"op":"d"}
{"before":null,"after":{"id":3,"name":"hudi4","price":3.33,"ts":2000,"dt":"20230331"},"op":"c"}

{"before":null,"after":{"id":4,"name":"hudi4","price":3.33,"ts":2000,"dt":"20230331"},"op":"c"}

{"before":{"id":4,"name":"hudi4","price":3.33,"ts":2000,"dt":"20230331"},"after":null,"op":"d"}

分别对应update、insert、delete ，其中 update 一次产生两条记录，先删除再插入。

需求

直接总结具体需求，关于 CDC 、Flink 写 Kafka 的细节等可以参考之前的文章:
Flink MySQL CDC 使用总结
Flink 读写Kafka总结

添加时间字段

主要是根据时间字段排序，根据官方文档：

Source 添加：

1
operation_ts TIMESTAMP_LTZ(3) METADATA FROM 'op_ts' VIRTUAL

Sink 添加：
1
operation_ts TIMESTAMP_LTZ(3)

字段类型也可以写成String , 两种类型对应的格式不太一样。
TIMESTAMP_LTZ(3) 对应 1970-01-01 00:00:00Z ，String 对应 1970-01-01 08:00:00.000 , 完整示例：

1
{"before":null,"after":{"id":1,"name":"hudi1","price":1.1,"ts":1000,"dt":"20260417","operation_ts":"2026-04-17 02:21:27Z"},"op":"c"}

相同主键落在同分区

添加参数：

1
2
'key.format' = 'json',
'key.fields' = 'id',

结果：

Blob 图片字段支持

首先要支持 Blob 类型的字段、然后要求转 base64

转 base64

参数：

1
'debezium.binary.handling.mode' = 'base64'

对应的字段类型为 String

图片验证

建表
1
2
3
4
5
6
7
8
9
10
11
CREATE TABLE `mysql_cdc_source2` (
  `id` int(11) NOT NULL,
  `name` text,
  `price` double DEFAULT NULL,
  `ts` int(11) DEFAULT NULL,
  `dt` text,
  `img` LONGBLOB,
  `insert_time` timestamp(3) NULL DEFAULT CURRENT_TIMESTAMP(3) COMMENT '创建时间',
  `update_time` timestamp(3) NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3) COMMENT '更新时间',
  PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
插入图片
1
2
3
INSERT INTO cdc.mysql_cdc_source2
(id, name, price, ts, dt, img, insert_time, update_time)
VALUES(1, 'hudi1', 1.1, 1000, '202660420', LOAD_FILE('/var/lib/mysql-files/query3.png'), CURRENT_TIMESTAMP(3), CURRENT_TIMESTAMP(3));

这里要求在MySQL服务器上执行，并图片路径，通过下面SQL获取，其他路径上传的图片都为null

1
SHOW VARIABLES LIKE 'secure_file_priv';

DBeaver 里查看验证

是个猫眼的图片，可以成功显示：

Kafka 数据验证

可以看到已经成功转为 base64 , 完整数据：

1
Partition:0     Offset:0        {"before":null,"after":{"id":1,"name":"hudi1","price":1.1,"ts":1000,"dt":"202660420","img":"iVBORw0KGgoAAAANSUhEUgAAAEIAAABKCAYAAAAPB4KFAAAgAElEQVR4nG2c2a9k13Xef2vtfWq6dYe+PXeTzabMQaQESjQ1mJJsC7KseIgiQ3acwEAAP+bFSIL4yU8G8j8EyEsCJ7KDvDhRLAGeFE2wLVOkSFlWTFOcxJ5v375zTeecvVce9t6n6rZSQJFdt86w9xq/9a11Sg7/7D+Y2AkQwDygaDQER6uOam2MOGVxvA8WEXNghkkgSoVfv8hwvMnx7h2kPqaixSxiIhiCAOARwIjpPhigIAIoIgZmlJdZOtNMEVGQOp8jgGBm+TNIvgbdXwArdzYMQ02JAkqAaEyC4+vf+T59MZ5/32WGBmq4tCAxkIgQQfKiY6SZTgmTo3QRsZXNgGlFfzCmmR4j7Rw1y5vIqyqbM2O59pWFlz92h1n+G0mIIpx+GWbxISGQ17vyEgGJIIbkNwgRxcToe+OFDz/L/uE+t/eOWNQBNakwU7AiVesWpEQ0tNA2aYFlkwiGgu/hnCfMp0houg1btx7JG7dOO6fWW/RoD22s06+xeoidPn0ptBXhnDpbDBMjSgAJIEoUj4hxdlzx3Aef4R/efY9JFDRKFoJVYA4xBTRd1gwsgEUwSUZjEcFh4nG9ATE0xHqGlo2IYuV8yuJXXeIhzZsloVoSWNJgzFqM+Sqa3w9bSHYtkoIk7T7vp1xTMCwdYyB4xARP4NGrF9jc3ubN23fR6B3gwDxiDsmbsOKLlhYTRdNtLd3c1NHr92nmJxDbtMQsCERPmW0y05A3Z4icNnvJvp8sKmZBRYyYP3sEtyKQ7sL52GIDiuQ3UvaloJLONNAI4DEclfM8/8FnuXtwiFdf5aUkDUXSwiUaYoqJSxLNko4iRAxVwUtkujhBJeQzi/RXNGZZOA+bbv4oop3GftK4SQI8JTCSxdnSYkRY3tXkIcMRLEI0UDFUAtHSfTUaZzfWePrpJ/DOj/KJSfqoITGmtymtU5CIEkEkWRsBidBM9ohtU3QDxE6gZgLi8kaz2a9sUIRsviBabFBXtFve4ZSApNtlcaNsoTkLLUNtOhcxJHqMmGJFVhgIqkmgj1+9hNfemIDHWUyLwrAYkQhYi7iVoJYvkNw30s4WuBwLjaL1oiU5vYEulSRzSBuIXRgVSkC0fGwJtOWUdI7ZqlBdvnfMVpEtrGSv5G3JVQCzwNK1yoUjG8MBXgfrNFKBNSiKxRQYY4zLAIRbaqLYdEy5WkSIJTN09p7eaXOxiw2n7l8ivQAmOduW4JhjiKUAaRQhRIrlFAyS/r+AsooutRhmki2vCFVLRO0EImI4Iqq9EVGrdJksVjHJaTBF8JSDhdUkJUIHetJGZLkBpDNxSamGn3yVwJmtp8MYRVDSvTHtFFGMJeeBtNnurNhd107l2oJddOXfyzUECyh+iFYjTLN/rtykbDhJ0udInL6LBiZL1acFFmkvF3T6czYC0fzOtmuSM0m5Hz9xXjJx6Vyp03ontByX8laXWUkzsDol56WrCpgzvFR9dDDEFgWSRqJEIpJTaUltsuK2lj2hBJ5isivAaFUjJQB2OT4t10QQsW4D2Y6Xb1E67awI2mw1SJJT5sqtVuwgId2Q/7J0veXyDCHio3qkqogYUSLEkMUhOWYUERtihklMFywSN4CwosmlDIp5iiwRacHDS4NeUZUlJNgJbUW4DwsDyQhYQMx3G00RxfK1JAstLgHuipikfI/hTSqq0RZz7SOhxcSBGK6KSSNaFhfpApIkXxMzRJq8B82mW2JESBAXw1mO89mERZRYhEQkaoQoKA4x6TZjxCT4EvQKIFNyIFXEHMV+VyLYCrxPisLkJ2RRsIiY4jFF+2cwN4J2BuZSgHMlBpQYah2EzVUIIoZaMbslpMVyxhEjrmQCIwC+K7bUIioRo02YI3owBxKANlnhyv2Tl6xaVLqnaCnGytalc7/OZU4FYyHjgyRc83hRQ3wfP1hDFg9QgdYEUe0kKOayeVl3rqJEhCCCWIMSiZqATZQqR3tBnNK6mKtPxWISlCMHXmtxtJTy1MQRJWZM4FBTDJcVsqx8k421rPq7yCrcKgCPpIBsxfaQywlKxOGNQHAVbriOHYHEpB0EpPAKMW0iOJf0YIJm34vEFNNEaHVI9BtIfx0/WMf3B/iqB84lY4mGBcNCTWzmtIsJ9eIYXRyi7RyNIV3Xcp1Q3APLQEk7gaX6Jbur9HJ8tSzA7Eqd0ABiZ7HSBeUSmwQPEXMOHa7TugqNixUAFJMfSooLZqWWCIiEVIfogKbXpxqdYbB2Hh2eg2pMdBVRoBYjZj5ATNDsq2otfYv0Yo3MD2kmD6gPd5D6CA1NJodIruralQyVk2hBkprtXk4jxlOAjZJlum9Xjkvy8QhEE3x/TOyvE8McVi5ipEzRQZLsIq3vE3vb+PVLrG1ehN6ISEWrPaKsApYVIqdgFNOl1ckQ1tapRucZbD9Cc7zD4miHZnqEr2s8RpQWI+Q06ZLLWYbSKpTUJjl+dLGkWEJxn4dwXQF9qhEvpJxvvTFu/RLN7BAfF3nbySSTLwVUhChC48foxmV6Z67D4CyNVMl0RSgMVtEdSAqCK68Ci0okNKto1RHUo9tjhuMrhJN92oNb1LMHSJwj4iAuMUhCvxGiW4KjDo2SLTlSCKe0cTvlFsVKzAyf0h4E6ePWLtD6W0hok5S7CO1AIq0KNtikv/04uvEIrR9j4rOx5kyxQrLkXZNqlfI6BdTzf9Nio1REHNIboJsjBqN15oe3scN3sHrWYQHJ6bDEESwssXd25QL6inunjcdT912+Ij6ZPphUSP8svfEVYjNFwixF3swXRDcgjs8xOH8dG52j0SFQZZNfZhPJET5aCWaW44l1+7esPS1VYXG9wo6JEX2fqBtUZ/sw6DPbuw3TQypbYCYEFSxqSt/q8nUfpuxKoC1FW3HVpJyExgSsxVspiCKYH1Ode5yTxSE620HjHJFAcCN0fJHRxadoh9u06hJ0MkumSVvAZ7aNdEMVwywkkNSh5byQXFIbCtqCCY4SkBP5I+owKmTjUQa9DRY77xKP76RUnVGlCsn8C72XA3MqxUOOoSlNF1wjXVngszAcHjwQcC4SRGB0hv7VDzK98zpusYsQ0bUrDC+8jzBYJ2ii8zSmTVsOZGlzGSxJocUCFltiqJJmNORYk8GaZJ1lLiS5fy7ISrAWBTwyOsf4cp8ZEA5votaAQjASJ0Ip08tLOoFIaRnkwiGbzxIXoXghgyWrUedoY4UbnWfjEUc72SHEyGDzGsGv0XTub911ihYkFzeW65M2NCxmU06Oj9jdOWb/YJ/p7BAlsL455vyFC5w7d5HBYA1XGc5lrhNbyRCJGw0IYhHX32Jw8Slm0eDoPVxc0OYSXopvdrv0KcBiOWA+xKJ3IDMJypOzhggEa7K8HLG/iQzGOJQ6DjAVTBqwkJBlof8togXMxJpmseDu/V2+89Lf8r3XXuXdd95lOoEYjECLCHjn8N5x/uw5nnn6/Xz0Y8/x7LNPs7GxhboS2AxwmCmm4CwQUKx/lurCk9T1CTrdwZU642HOI5M9smIZZHJniSlCFlJEju+9Y0KDqSWCE0OjEjPXKBYxc1n9bQJThciNhQ5rCU3Dg/v3+fo3vsE3vv1tbt66Td3WhDZAyEWWSPJtUuGlonjvGPeFJ598gs9+9nO8+OKLbG6tgyghKmgPpEYtAo7WHC7UuOObzG//kF59QIiFJV9xDfNL6K0li8VOeUthCGqKHN972yBiokRzpYLAJBVZLgZiBkJi1vlVLBkgGovpMS9/92X+55e/wutvvkPdtik4xZqeBAYaGFSKmhBjoDWjNmUShAboo6g4Rmsb/PTzz/PFL36Bp59+KvVNTHEScnWwgh6bKXJwg/ru6/hmCtIA7ekYYQUDrcaOU+A6fy/I8b03zSB5ScwFj65qXjBtwEBj6YZFggViiOztPuBPv/pV/vzP/4L94wVNNCQsGDPj0ijyxIV1rqz32Bp41irPoFIWBvdmLe/sn/D2ziF3ThyL2Kc1h3fC9WtX+c3f/HV+5pOfpD9cR3OBhuReR0YufnHI9PYbuMP30DhPwpDCZRT3SGVVZwEFea7CbjHk+N4bhqS6XizzkiX/5wCjJdBY6gUEa2iahh+/+x5f+sP/zms/eIW2jlgDfRqurkWeuzDkiU3Y9jVjhWFV0a8EJ6lIa1yPue+x3wRev9/w8o+PeHMPJnGAacuF8+f54q9/kV/51V9mOBhROIkV8g5vLWGyT3PzVdxiB4mlm2aZ1wgEKTxHEkxH3YihufmsSG5T5y+WeK9Q8y1Ki0YhqhJyCzDGyK1bd/kv//lLvPZ3f0/bLhiFhqujimcur/HE2Yr1MGUUWyoD3x/h+h6TFpEWxehpoCczNgeRq496PnDpEf76vSnfeuMBe23Fzu4+X/qj/8GwP+Czn/sMvV6fVIZlDIMQqZDhFn7jMs3eIS7OU2rOyktxMmLR5T3lwFn+Tkg7NcX93u/+298vUjz96nhipAMwqdDZ3d3lD770R7zy6veop0eshRnPbA/42NU1Hl0L9J1w0ig3Tow3DgJvPKjZa5TW93G9Pq7qoaqoCOqEPoGxg+vnhpwZ97m7e8A8CItgvPXGm1y7do0rV692rp9eSbciUDlHO5tBO0GsJWaMko7VnNpXYgcFjJe6B3z6mHlIgM4aigFqihuZkJ3OZnz5j/8XL//td5kc7bLt5jx/cczT5zwjJhzXjht7Na/fPWR/YdQoYhW9GwvGvuH6+REfuX6GZ854tipDxCEKjoYzVvPJa1tU7hG+/MpNbtQVD/YP+K//7Q+59th1Ll2+uLJOWy51sElv/QqL+X3U5om+k5T9MEDbjHtKTCjVedqjmuF+73f/3e93/ENXuCzZxxxzEIS2bfnGN77FV/73Vzl+sMs4nvCRqyM+dGmNNW1p8Pxot+G1WyfcbzxT8SxQGioW4ji2iluTyFv3jzlatKwN+4yHFapGdB6nSs/g3HqP4aDPOzsnTKLn8PAAEeEDH3wWX2mn7TJfEvH0RFgc30HDPEMKQczhzGGamKxSuaZNRcQiGo24yPipsDpdtM0YrDRnnQkSjJs3b/Hlr3yVnQe7DLTlqe01ntwcok1LHQfsLvq8udtw2A5orb8sgSUQgQZhJj3k6gd4/xd/h8m1F/nxYkgrAxyOKB5zxlhnvHCl4mOP9OjFhrZp+NrXvsabP3orl+IrTSWBKEB/gBudwcQnLNQNrJzu0Ui2AEKgnc85OTjg6MEemoqghOeXnSBbisIMYqCup3zzm9/kvfduIO2Cs33j+tlNkCFHNuBAx7x90LLTCDOU0HEDRrCKNlaYOXq+4pd+9Z/y8//sX/Kzv/VvsMd/ltv1EBWPd2AaEQdn+i2ffnqbx9cNT8PR4T7f+Po3WSxCInbymIDlEYDgKvobFwnShzQbk9ZfOAlLVq1mSAg0kymzgyPa4wk0AS3wwkqfcVktd6kqSODW/du8/PLLyKJlU+GJsxtcvniJ9//85/ml3/l9Pv3b/554/nHmWhE6JsoTJDHeackt/dGQx598AvVgwxGPfeKXue8vsvADEJdaNargPJc3HD/z5CZ9qREiL7/yCjs79/PGXHbhJJSgDr92Htw6y45bwCSgpmgUaCL1dMbR7h6TvQNkUeNizGQ0pVUel/V6ZnjKDdvGeOWVH3Dr1l0qM66N+1zd3uTR5z7Chz//G2w88xEufPDjvPgLv4Lzw8RwW9KLRIdai7M6DWmYw7thgu3a5+zl65x54nn2wxBxHqdVStWVMHAtLzw24vHN5GJ7B/u8/OrLWKxzZ7sj9VPLsj9AB5tEekQ1grYoDRID7WLBycE+J/v7hMUMD9laIzmkr+i+VJKF8Mwh9vjwhFdfepkwn7FVNbzv4oiN7W2efvEXCGtnCdLHVz3OnztH5V2uBFugwZnhLGR+QJhP5hztHWLRE8yDVJy58lPs1RWoy1xJahb1HJwfwAuPn2egENqW733/Vep6mmCxFZInYAYtRjU+R9RRLt8DVs+ZH+4z2dulnZygTY2LiQ6IQNQUfDWZ2CrZmtklAhCIsebmzXe4+e6PGLDgkXXj8pZn4/Jl1i4+Ajh8NDQaTdMQLWDSZrjbJKhOauSIBGK74LVXvst8dkIkEDQwj553d09oMm/gTKjMIyg9FZ64tMEZ36DW8uZbb7N/eJSoGyuKizgTjB5+fIbgh8RGiZOG6cER4eQY1yQ36HYqkuKuSA6wKwGyg6dS4HWgDQt+8HevMpscs17B1S3P+iAyGA/Au+xfAbThZLJPjE1mrySV59lPUUspK7Z86//8Ba++9FeE+phFfcxfv/QKP7q9y6wtRZLlWS2HE8e1MwPet91HrWX/4Ij3btxKAVFyjBDrnCQ6T2vQTBfUh1PiItDGkFzHOagq6A9g2EfXBjCskN4An7uoXVW5DJVJGLPplLfefgexyJl+4KcunmV9GJFwiIYpVo0xdRiBg8k+IdaZn0h/i5p7qHmDEgPTgz3+4D/9R/7+tb9BJPCtv/wrztoB0/o8m4PCLQQiHqgYS81TF9f5u/v7zGPg5u3baarHpatG0ZTiAVXDwoJmcow2EZMeDAeI9/SqHuo96jV3C41oLdr28UuXkMwtuFRuA5hxeHDA7Ts7VBZ4fMvx2BmPdzUns7vY0U1kuE2wHphwcHCAWcRiSGOKuiJWy4k8gsSGO+++y5+89wZIS7QeWkWOpg2XBoKQYHLIAdsTubzVoyctjsDOvXuEaGiVbdh8CnEacBKIzQznoDcaoNUIhikISx6sTYRuwCSTSq7CU1rmWWeSjcxMsGjcubPDg90DztDy9OUzbFUNqhENR9z9/l9y7RPruME55nVg7+59Yky9j9J3lFL5pbyUMGuUzBFkpkCVRWsczGaknpPl/mdEJMHj7fGAgQa8NRwfH3bTNKkRvexk4gQ/8LhRj54fgBuCC3lXrhsDSBWqZSwSU8uPQrjgWS1KDMetW3do6paz6yMunVmn52YIsOECk3de4mZzzKUPfYo7DwJ3fvweFgZE2jwukCxDJWRm2+UOu1vyh6IYFS0wCy24RAcgisYUYA1lbegZeMU1gbapy1bADJemCgiieNdD+wN0MMgua6jETgmRlJWwBLlS2oyl6MqmTILZiZ2GNrTcvbdDDDUX1iu2RxU9WpDUvahoWNz4If/w7pt8590Jd9/bIQ2HFmySLCvmRoxkus9osyYklcjaEiXSxthR/GpJEZE08NHz0PdA4/HOp9YjeTZDImk62GOxwukQV1VYTIE8CaCQO2Wis8xgJB/w5UtWKrMyv9C0NQ/29lAaLm6uMXQBDZmviGmBI4y+Rtamx1g9x6x3SgjdwFkeQRYp0zVLWC80HRqMwXBudXQsp1RNgNNw9PsDyqAqpplLiakN2LYQu95WNwwnBSZk5r0UkrLk50IiKbo22bIsr9uGk8mEngTOjHtIaZJk8zKLBeCyMfA4QueDaSMO6+ab8n+tzGmvuKA5xCKV+mS23bhy6Cpt6IE4UGG8uYnlGUvFJ75EDaMltHMILWql+5UdQlroRoysG9cqE30+TafYEkrkEGYWaeuG6cmEvkbWB4pmU1wNrKmcNbZGQ6oyWkjiCS0zz2WkTbrzugonfRbFCwyqCtHUZ6VDpxGNSrAqdTFVuXTpUh5kyfW4FaqgIdbHqDXJErsxvUVKtwSWG82ukUtz7UAUIGbLOToz2rqhni3oO2VYaWrhJTGWJeQ9KmvDPv1qhQ7r9G7dtsVW89IKZySJuhtUnjKalMw6DykJNG2qJ3qV55GrV1AlPynQXQTCnHbyAAl1Eo7lOLTi9il9hs79yxdaAiRRMmu9pK9CCLRNixNXSIuuqC8fjRSxx8OKzaHHZZ8sPEAZVU1+WSa2sxuS+xG0rPVhPHB0X3Ud7MQx7s9qZq1x9tw5Lp2/CFb4xtCd42ONLI4RqxNHQWn85lHJNLazFELHTqaeWjJfPJbTZ5IitCHQhJYQhRBT6630OApXYWIggbW+cWmjYkCep8Jl7jAPhuVMcmpYVPIQitRsb3g213qoFnPP5IsJLZ57B1NmQXn22ecYr2+m6J8zgBlINLSew+wIsUWmFw2xCDELwUoi6PaflyKp1jDTzozKqHHMQyGu16M3HNOYA6m6m+fz878aBrrgqavbjKTFYaDlEQcQyWOCp0aSoMB4peXKxU36FRgF4GXSBaURz+29E6Qa8sILP0O/P8rfW5ZXIlyakyO0nYEFAjF34svIUR5H7Oar6PoaIpqQZRFMOinVBiaJIBltbHPp8nkm4T5BAkqT+gRiaEycAwLeIk+eX+fcsMfhpLCjZaCjWFpnb7lQiiiBoTOevXiOCiGKw2WfTkOwxjRE3t5fcOGR53j/00/inHRtyJjnuXphwmy6g1BnHJJnLfI9lZQZY7bE0rZMdVBEC/BNGs6NHROcCJUTxsMhj1x7kjt7C0wqNDYoLdadlybgHMaldcdPXVqnp8nR+lZmJfNISIqWaVkC0SWBXd7o8cT2ekq/pLZB0GQdgcCt/Sn7bcVHP/5zrG+M6Z4qysOixIjNDgmzAzoMmcFyFC20U7K2PIKQ3CSJyKwtMaKYjsvxMOIsaWprWHH57GVa2WTe9HHiUodIBVNBNDXwRWDNt3zo+jZnqkBlEZ+5Q+m0H3FEnIBYD40VA+3x4WtnODuMVDQ4WoJYmtUgEk34x9uH9LYu8NEXX8S5agUoJSa6Cg31yT60dRJALFOgbRqbpowm5mGDmCiC8vRZWpuspjNNTRxSShsNeqwNegx6a2xeeoK7x0KUHqY5hWp6SEw1aaFnLR+4tMZzlwesuQDqUU3aRyqcORwtQno2ZIDxyFj42JPnWdNFGv4gzW5iqd96f2q8eb/m+Rd/jguXLuTIolh+OkcJuHpKffwAjXXuZbhUBVubLFzL8ElyF83PnSUnTQ3w1BmLUJodmcAn4nC9If3ROnuzBZef+WluHBlttYaIoFJQevYzTfXEZtXyueevc33L6LuIahrvKegAgaiCc4GzvYbPPneZRzY9ajVlflolItbQSo/v35jQO/c4P/+Zz1E5TQqQPFpExIWG5mgHFkdobFbSJZRheIslJqQVx5LGxQgCEZcj3al8kT+JR3tD1jbPcu9kyoWnnuXQb7BXe1CPy8cGdUQqyIENiTy66fnVDz/G9XHL2Df5CZkUE6Ib4NSz7ua88Ng6Lz62xVAakFwYZegepMedacXre45PfO4LXLhwORH4OVMpisYGme+zOLiNj4sU6UpKLfgIw2X37ICD5SMsIjF1/VODR/MEXJ6JUNJkiTrHmfPn2D3eY3hmi0ef+zjfvzWhlsRKJcyT/C3VCAlOD7Xl+avr/MbHHudDF4RzvZahtvS1YU3mXKyO+OhjymdeOM/mYIbPaDVK7lWI58DW+es3jrjyzKf48Ec/hfd9ltPhCdZX7YJ67wbS7KNWr2CTgiqXzWzJRZ9ZYsmkaWlPJtSHRywODjNDlUGSWMxcYYJWUSKXLp5l8b3vUc+O+dAnPsNXfvA33Dg64tp4RCUBtRafNyGkLpLRMtSGD13pc+XsE7y903LjwQmLtmVrJFw/V3H94pBxXxLFXx5CMYcQqcXz2nvHTMeP8Wtf+C0G61upRypGoMGIuBipj/eIJ/eQOAELiK483SGpkCvD6QmsChYioW4I0xntfEGoGwyX+Qijw/iRNAco+XnPS5fP0dPA4e49rn/wOZ791Bd4+Wt/yNrj61x0RymVms/VaEt5qMVIc0/nesbZR/t89H1jIhGVmh51moSLSrAeJousTVi4If/33py3Z0N+6V/8NhcuXcmjPxBEUmALc6iPWDy4Ta9JowAR6DCRFZ4h0Y6mpNGl0DKfTJgfH0LboiEdHxhSRmAofFqSSX5yT5Tt7TOc395m5+5diMazz3+cjSc/xnfvNezqGk3PIb5FtAZvRGdEjUQNBGdQKZVr6NuUgUzoyzw9oxEzsNEEs41IS48f3Jzy97s9fvE3/jWPPf1+XGX5AdsWk/QQTNXOmd9/B5vdT5ZgnvSocyJ0EtVHCoTiwRQJgfrkhPnRA6ytCWa0rqLprcHWhVyGr8DQ5aBVqm0H/SGPPnqZe/du04SGwdoan/wnv8af/cmEb9/+AZ98dIPL/RmubbNfll8VaFCLaWisPCMhDitslZbJlUB0yrQZ8Oo7R9zTi3z2n/8rHnv/c0gFpqV0jmiM+MWMevcW8egWPTtJWCBWCI6g5PomBUcNqZFjUWkXLfPpHIseqXq4/ia9zfMMxucY9IfI8c4b1mH/DEtLNWKWNHb79rt87evf4vOf/3U21taJBnsHD/j2n/0xh29+hxcuO66ve0ZxgbZz0JBGCS03XmSQofiyNBfnMPXU0XHzuOYfdx2DRz/C85/+ZbYvXkFcfljFRaKmbnyvmdHsvke9+w5Ve4gyxwhoHCb4rAmLQsRHcNGwEJm2numiRao+w/EW1XAd9WkCp1k0hKO7yPHO6yvlWO53rnAFhhFtzoP9PcbjLQZ+mH+YQplOJ/zw+y/xw5f+lM3mAc9uKdfWWvpMQBqIAWeApeF1yxxadD3mMmJ3Jvz4/pTZ6CKPPf9pHn32IwzWh3gptKoHaYhiVM2CsHeD+sFbuMUhLrak8Z8GZC0n/QAWMoL2RPoJ1PU20V4fVw1omprFZMLi4AFhcozNJ7h4ghzd+8fE0XS85WnSBDKmzwWUUX5VACKpi3Rv7x5vvPoSu//wCqPpHR7dhO1RZK0i9T5jRTBlZspRGzmoHfvtkMHZ61x7+nkuvO+plBlcBZpCtssAL0jEtzOag5s0u+/gFwdozEE5K0qlRxmgjyjmB0hvhB9tob5PnAfmx3s0h3epD+9hswk+tHmwIsVHObr3IyvExQpTcepl5nLRG3PPIkNSawmWgFhoAge7+9x+9y12b/6I9mgHCRNoa7wY2uvDcINq/RxbFx7hwpXH2Dp7Ed8bYi49TA+JhFEzvAWEmjZMCA/eowvemQIAAAKwSURBVD64SdVMcE2uHyQ9tBIy0hTfR/062h/jeyOcKnU94XB3h8XuLnF6iGuOce0Eb23elyfiic4hx/fesp+0glUaC8QSrxBTMzMFwZiekWgz3CYmaG4mxDYSm5rQTIlhiriA8xXq+3g/xLn8nCma23Wl+MlcoimEFpnvUe+9iZ7cQNomgb3ugX9H66CtFDdcw/fXcTJC8dh8wcnuLaZ7d4jTfVwrqLVgC8yajDLJjJVD1GUWu/SIu+c7VwOnZrKk/CRCJk5UsqVoyjqS+UgRxHt8lYgcGHeMYDeHQaHYS+Va/Dog1mJhTjw5oN75MdVsD7UF5QGU4Byt95hfw43WqfpDxCJtXdMu9mF2xGJ/l/poH+oaZ21qFJnkgN1PRV3Zk7SALz8HJMUH8oJzMb+EaWnhK1ayyocuDyw0+ApxK0D+pSLK5wzly7lpojdiEgjzI+qDe7THD5B6SqMVUddRV4FWSG9Erz/C+T4xROrZlNCc4JopujikPdojTo7QNiQQZZpxkVFGmEUyP5u5EgCfuP4VtyjgamWwrHOdbjiMh75fnXUuP6sST5/TcR55LsvKMQ6jl6wkNrR1i4hnMD6LbF5Gqj5VpQmDuIpgAqFmfrJPe7BLLy7Sc6OzI2R2QDg5ItY1IWpK30TKc5+SOVAthJIV5RaI/VCWYJXgzJtLr+I2q1ZQ9mkdc7S8nqwclwVhCeysnh00C1YGVGsXkfF2/sWKDMCsIcRA07aEGFCnyNoWfe9pJye0+7swB5m1EFL8UgIxN6M0B3ST9JTgclwkNZxd1DIf8dC+fyJwRn7yoFWL+P/s+9Qf07twBd3fs/UpNVaOcVUifS3iTQhNpGkbEKhcj14/EwCDLWxoVONIf7zPbPcG9W4fibeQuI+GOpPQPgsi1xuSOdnuYVildRX/DwyIBNmWraJ7AAAAAElFTkSuQmCC","operation_ts":"1970-01-01 08:00:00.000"},"op":"c"}

base64 验证

编码转图片在线网站：https://www.sojson.com/image/basetoimage.html

到此图片功能验证成功。

完整SQL

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
set yarn.application.name=cdc_mysql2kafka;

set parallelism.default=1;
set taskmanager.memory.process.size=3g;

set execution.checkpointing.interval=1000;
set state.checkpoints.dir=hdfs:///flink/checkpoints/cdc_mysql2kafka;
set execution.checkpointing.externalized-checkpoint-retention= RETAIN_ON_CANCELLATION;
set pipeline.operator-chaining=false;

CREATE TABLE mysql_cdc_source (
  id int PRIMARY KEY NOT ENFORCED, --主键必填，且要求源表必须有主键，flink主键可以和mysql主键不一致
  name string,
  price double,
  ts bigint,
  dt string,
  img String,
  operation_ts  String METADATA FROM 'op_ts' VIRTUAL
) WITH (
 'connector' = 'mysql-cdc',
 'hostname' = 'ip',
 'port' = '3306',
 'username' = 'username',
 'password' = 'password',
 'database-name' = 'cdc',
 -- 'server-time-zone' = 'UTC+8', -- MySQL 时区，根据实际情况添加
 'table-name' = 'mysql_cdc_source2',
 'debezium.binary.handling.mode' = 'base64'
);

CREATE TABLE if not exists test_kafka_sink (
  id int PRIMARY KEY NOT ENFORCED,
  name string,
  price double,
  ts bigint,
  dt string,
  img String,
  operation_ts String
) WITH (
  'connector' = 'kafka',
  'topic' = 'test_kafka_sink2',
  'properties.bootstrap.servers' = 'ip1:6667,ip2:6667,ip3:6667',
  'properties.security.protocol' = 'SASL_PLAINTEXT',
  'properties.sasl.mechanism' = 'GSSAPI',
  'properties.sasl.kerberos.service.name' = 'kafka',
  'properties.group.id' = 'dkl',
  'scan.startup.mode' = 'earliest-offset',
  'key.format' = 'json',
  'key.fields' = 'id',
  'value.format' = 'debezium-json'
);

insert into test_kafka_sink select * from mysql_cdc_source;

debezium.binary.handling.mode

官方文档

https://debezium.io/documentation/reference/1.9/connectors/mysql.html#mysql-connector-properties

解释

这个配置项，用来指定数据库中的二进制列（例如 BLOB/BINARY/VARBINARY），在变更事件中应该以什么形式表示。可选的配置值如下：

  • bytes：将二进制数据表示为字节数组。 (默认值)
  • base64：将二进制数据表示为 Base64 编码的字符串。
  • hex：将二进制数据表示为十六进制（Base16）编码的字符串。

字段类型映射

具体的二进制列： BINARY、VARBINARY 、TINYBLOB 、BLOB 、MEDIUMBLOB 、LONGBLOB

上面字段类型的对应是针对 MySQL 的，每个数据库类型是不一样的，比如 Oracle 只有 BLOB ：

验证

验证其他两个配置项的结果：

默认（bytes）：

1
java.nio.HeapByteBuffer[pos=0 lim=8949 cap=8949]

看不到具体内容，因为它打印的是缓冲区状态，不是数据本身，所以无法直接转图片：

hex

hex 比 base64 长度更长，以同样的原始二进制数据为例：

  • 原始数据：1 字节（8 位二进制）
  • hex（base16）编码：需要 2 个字符表示 1 字节（每个字符代表 4 位），长度是原数据的 2 倍
  • base64 编码：需要约 1.33 个字符表示 1 字节（每 6 位二进制对应 1 个字符），长度是原数据的 4/3 ≈ 1.33 倍
1
89504e470d0a1a0a0000000d49484452000000420000004a08060000000f0782850000200049444154789c6d9cd9af64d775de7f6bed7d6aba7587be3d7793cda6cc41a4044a343598926c0bb2ac78882243769cc040003fe6c54882f8c94f06f23f04c84b0227b283bc38512c019e144db02d53a44859564c539cc49e6fdfbe734de79cbd571ef6dea7eab65240915db7ceb0f71abff5ad754a0effec3f98d80910c03ca068340447ab8e6a6d8c386571bc0f161173608649204a855fbfc870bcc9f1ee1da43ea6a2c52c6222188200e011c088e93e18a02002282206669497593ad34c1151903a9f23806066f93348be06dd5f002b77360c434d89024a80684c82e3ebdff93e7d319e7fdf6586066ab8b4203190881041f2a263a4994e0993a37411b195cd8069457f30a6991e23ed1c35cb9bc8ab2a9b3363b9f69585973f768759fe1b4988229c7e1966f1212190d7bbf21201892086e4370811c5c4e87be3850f3fcbfee13eb7f78e58d40135a93053b02255eb16a44434b4d03669816593088682efe19c27cca74868ba0d5bb71ec91bb74e3ba7d65bf4680f6dacd3afb17a889d3e7d29b415e19c3a5b0c13234a0009204a148f8871765cf1dc079fe11fde7d8f4914344a168255600e3105345dd60c2c80453049466311c161e271bd013134c47a86968d8862e57ccae2575de221cd9b25a15a1258d260cc5a8cf92a9adf0f5b48762d928224ed3eefa75c53302c1d632078c4044fe0d1ab17d8dcdee6cddb77d1e81de0c03c620ec99bb0e28b96161345d36d2ddddcd4d1ebf769e62710dbb4c42c08444f996d32d3903767889c367bc9be9f2c2a6641458c983f7b04b72290eec2f9d862038ae43752f6a5a092ce34d008e0311c95f33cffc167b97b7088575fe5a5240d45d2c2251a628a894b12cd928e22440c55c14b64ba384125e4338bf457346659380f9b6efe28a29dc67ed2b849023c253092c5d9d2624458ded5e421c3112c423450315402d1d27d351a6737d678fae927f0ce8ff28949faa82131a6b729ad539088124124591b0189d04cf6886d537403c44ea06602e2f246b3d9af6c50846cbe205a6c5057b45bdee19480a4db6571a36ca1390b2d436d3a1731247a8c986245561808aa49a08f5fbd84d7de9880c7594c8bc2b0189108588bb895a0962f90dc37d2ce16b81c0b8da2f5a22539bd812e952473481b885d18154a40b47c6c09b4e594748ed9aa505dbe77cc56912dac64afe46dc95500b3c0d2b5ca85231bc3015e07eb34528135288ac51418638ccb00845b6aa2d8744cb95a4488253374f69ede6973b18b0da7ee5f22bd002639db96e0986388a500691421448ae5140c92febf80b28a2eb51866922daf08554b44ed0422623822aabd1151ab74992c5631c9693045f0948385d5242542077ad24664b901a4337149a9869f7c95c099ada7c3184550d2bd31ed14518c25e781b4d9eeacd85dd74ee5da825d74e5dfcb35040b287e8856234cb37faedca46c3849d2e7489cbe8b06264bd5a70516692f1774fa733602d1fcceb66b923349b91f3f715e3271e95ca9d37a27b41c97f25697594933b03a25e7a5ab0a9833bc547d7430c416059246a24422925369496db2e2b6963da1049e62b22bc0685523250076393e2dd74410b16e03d98e976f513aedac08da6c35489253e6caad56ec2021dd90ffb274bde5f20c21e2a37aa4aa88185122c490c521396614111b6286494c172c123780b0a2c9a50c8a798a2c1169c1c34b835e51952524d8096d45b80f0b03c9085840cc771b4d11c5f2b5240b2d2e01ee8a98a47c8fe14d2aaad11673ed23a1c5c48118ae8a49235a1617e90292245f1333449abc07cda65b6244481017c3598ef3d9844594588444246a8428280e31e93663c424f812f40a2053722055c41cc57e5722d80abc4f8ac2e4276451b08898e23145fb673037827606e6528073250694186a1d84cd550822865a31bb25a4c572c61123ae64022300be2bb6d4222a11a34d98237a300712803659e1cafd9397ac5a54baa76829c6cad6a573bfce654e056321e383245cf37851437c1f3f5843160f5081d60451ed2428e6b2795977aea244842082588312899a804d942a477b419cd2ba98ab4fc562129423075e6b71b494f2d4c41125664ce050530c9715b2ac7c938db5acfabbc82adc2a008fa4806cc5f690cb094ac4e18d4070156eb88e1d81c4a41d04a4f00a316d223897f4608266df8bc414d344687548f41b487f1d3f58c7f707f8aa07ce2563898605c3424d6ce6b48b09f5e2185d1ca2ed1c8d215dd7729d50dc03cb40493b81a5fa25bbabf4727cb52cc0ec4a9dd0006267b1d205e5129b040f11730e1daed3ba0a8d8b150014931f4a8a0b66a5960888845487e880a6d7a71a9d61b0761e1d9e836a4c741551a01623663e404cd0ecab6a2d7d8bf4628dcc0f69260fa80f7790fa080d4d268748aeeada950c95936841929aed5e4e23c653808d9265ba6f578e4bf2f10844137c7f4cecaf13c31c562e62a44cd14192ec22adef137bdbf8f54bac6d5e84de884845ab3da2ac02961522a76014d3a5d5c910d6d6a946e7196c3f4273bcc3e26887667a84af6b3c46941623e434e992cb5986d22a94d426397e74b1a45842719f87705d017daa112fa49c6fbd316efd12cdec101f1779dbc924932f0554842842e3c7e8c6657a67aec3e02c8d54c97445280c56d11d480a822baf028b4a2434ab68d511d4a3db6386e32b84937dda835bd4b307489c23e2202e314842bf11a25b82a30e8d922d395208a7b4713be516c54acc0c9fd21e04e9e3d62ed0fa5b48689394bb08ed4022ad0a36d8a4bffd38baf108ad1f63e2b3b1e64cb142b2e45d936a95f23a05d4f37fd362a354441cd21ba09b2306a375e687b7b1c377b07ad66101c9e9b0c4112c2cb17776e502fa8a7ba78dc753f75dbe223e993e985448ff2cbdf115623345c22c45decc174437208ecf31387f1d1b9da3d1215065935f6613c9113e5a096696e38975fbb7ac3d2d556171bdc28e89117d9fa81b5467fb30e833dbbb0dd3432a5b602604152c6a4adfeaf2751fa6ec4aa02d455b71d5a49c84c604acc55b298822981f539d7b9c93c5213adb41e31c91407023747c91d1c5a76887dbb4ea1274324ba6495bc067b68d744315c32c2490d4a1e5bc905c521b0ada82098e129013f923ea302a64e35106bd0d163bef128fefa4549d51a50ac9fc0bbd9703732ac5438ea1294d175c235d59e0b3301c1e3c10702e1244607486fed50f32bdf33a6eb18b10d1b52b0c2fbc8f30582768a2f334a64d5b0e646973192c49a1c502165b62a8926634e45893c19a649d652e24b97f2ec84ab016053c323ac7f8729f19100e6fa2d680423012274229d3cb4b3a81486919e4c2219bcf1217a178218325ab51e76863851b9d67e311473bd921c4c860f31ac1afd174ee6fdd758a1624173796eb9336342c66534e8e8fd8dd3966ff609fe9ec1025b0be39e6fc850b9c3b7791c1600d5719ce65ae135bc910891b0d086211d7df6270f12966d1e0e83d5c5cd0e6125e8a6f76bbf429c06239603ec4a277203309ca93b38608046bb2bc1cb1bf890cc638943a0e30154c1ab0909065a1ff2da205ccc49a66b1e0eefd5dbef3d2dff2bdd75ee5dd77de653a81188c408b0878e7f0de71feec399e79fafd7cf463cff1ecb34fb3b1b185ba12d80c709829a6e02c1050ac7f96eac293d4f5093addc1953ae361ce23933db262196472678929421652448eefbd634283a925821343a31233d72816317359fd6d025385c88d850e6b094dc383fbf7f9fa37bec137befd6d6edeba4dddd6843640c8459648f26d52e1a5a278ef18f785279f7c82cf7ef673bcf8e28b6c6ead8328212a680fa4462d028ed61c2ed4b8e39bcc6fff905e7d408885255f710df34be8ad258bc54e794b61086a8a1cdf7bdb2062a24473a582c02415592e0662064262d6f9552c19201a8be9312f7ff765fee797bfc2eb6fbe43ddb62938c59a9e04061a18548a9a1063a035a33665128406e8a3a838466b1bfcf4f3cff3c52f7e81a79f7e2af54d4c71127275b0821e9b29727083faeeebf8660ad200ede918610503adc68e53e03a7f2fc8f1bd37cd207949cc058fae6a5e306dc04063e9864582056288eced3ee04fbffa55fefccfff82fde3054d34242c1833e3d228f2c48575aeacf7d81a78d62acfa0521606f7662defec9ff0f6ce21774e1c8bd8a7358777c2f56b57f9cddffc757ee6939fa43f5c47738186e45e47462e7e71c8f4f61bb8c3f7d0384fc290c26514f748655567010579aec26e31e4f8de1b86a4ba5e2cf39225ffe700a325d058ea05046b689a861fbffb1e5ffac3ffce6b3f7885b68e58037d1aaeae459ebb30e4894dd8f63563856155d1af0427a9486b5c8fb9efb1df045ebfdff0f28f8f78730f26718069cb85f3e7f9e2af7f915ff9d55f66381851388915f20e6f2d61b24f73f355dc620789a59b6699d70804293c47124c47dd88a1b9f9ac486e53e72f9678af50f32d4a8b4621aa12720b30c6c8ad5b77f92ffff94bbcf6777f4fdb2e188586aba38a672eaff1c4d98af53065145b2a03df1fe1fa1e93169116c5e869a02733360791ab8f7a3e70e911fefabd29df7ae3017b6dc5ceee3e5ffaa3ffc1b03fe0b39ffb0cbd5e9f5486650c8310a990e1167ee332cdde212ece536aceca4b713262d1e53de5c059fe4e483b35c5fddeeffedbdf2f523cfdea7862a40330a9d0d9dddde50fbef447bcf2eaf7a8a747ac8519cf6c0ff8d8d5351e5d0bf49d70d228374e8c370e026f3ca8d96b94d6f771bd3eaeeaa1aaa808ea843e81b183ebe7869c19f7b9bb7bc03c088b60bcf5c69b5cbb768d2b57af76ae9f5e49b7225039473b9b413b41ac25668c928ed59cda576207058c97ba077cfa98794880ce1a8a016a8a1b99909dce667cf98fff172fffed77991cedb2ede63c7f71ccd3e73c23261cd78e1b7b35afdf3d647f61d4286215bd1b0bc6bee1faf9111fb97e8667ce78b62a43c4210a8e863356f3c96b5b54ee11befcca4d6ed4150ff60ff8afffed0fb9f6d8752e5dbeb8b24e5b2e75b0496ffd0a8bf97dd4e689be9394fd3040db8c7b4a4c28d579daa39ae17eef77ffddef77fc4357b82cd9c71c731084b66df9c637bec557fef757397eb0cb389ef091ab233e74698d356d69f0fc68b7e1b55b27dc6f3c53f12c501a2a16e238b68a5b93c85bf78f395ab4ac0dfb8c8715aa46741ea74acfe0dc7a8fe1a0cf3b3b274ca2e7f0f00011e1031f7c165f69a7ed325f12f1f444581cdf41c33c430a41cce1cc619a98ac52b9a64d45c4221a8db8c8f8a9b03a5db4cd18ac34679d09128c9b376ff1e5af7c959d07bb0cb4e5a9ed359edc1ca24d4b1d07ec2efabcb9db70d80e68adbf2c812510810661263de4ea0778ff177f87c9b517f9f162482b031c8e281e73c65867bc70a5e2638ff4e8c586b669f8dad7bec69b3f7a2b97e22b4d258128407f801b9dc1c4272cd40dac9ceed148b60042a09dcf393938e0e8c11e9a8aa084e7979d205b8ac20c62a0aea77cf39bdfe4bdf76e20ed82b37de3fad94d90214736e040c7bc7dd0b2d3083394d0710346b08a365698397abee2977ef59ff2f3ffec5ff2b3bff56fb0c77f96dbf510158f77601a110767fa2d9f7e7a9bc7d70d4fc3d1e13edff8fa37592c422276f29880e51180e02afa1b1709d287341b93d65f38094b56ad664808349329b38323dae30934012df0c24a9f71592d77a92a48e0d6fddbbcfcf2cbc8a26553e189b31b5cbe7889f7fffce7f9a5dff97d3efddbff9e78fe71e65a113a26ca132431de69c92dfdd190c79f7c02f560c3118f7de297b9ef2fb2f00310975a35aae03c97371c3ff3e4267da911222fbff20a3b3bf7f3c65c76e12494a00ebf761edc3acb8e5bc024a0a668146822f574c6d1ee1e93bd036451e362cc6434a5551e97f57a6678ca0ddbc678e5951f70ebd65d2a33ae8dfb5cdddee4d1e73ec2873fff1b6c3cf3112e7cf0e3bcf80bbf82f3c3c4705bd28b44875a8bb33a0d6998c3bb6182eddae7ece5eb9c79e279f6c310711ea7554ad59530702d2f3c36e2f1cde4627b07fbbcfceacb58ac7367bb23f553cbb23f40079b447a443582b6280d1203ed62c1c9c13e27fbfb84c50c0fd95a2339a4afe8be549285f0cc21f6f8f084575f7a99309fb15535bcefe2888ded6d9e7ef117086b6709d2c7573dce9f3b47e55dae045ba0c199e12c647e40984fe61ced1d62d113cc83549cb9f253ecd515a8cb5c496a16f51c9c1fc00b8f9f67a010da96ef7dff55ea7a9a60b115922760062d46353e47d4512edf0356cf991fee33d9dba59d9ca04d8d8b890e8840d4147c3599d82ad99ad925021088b1e6e6cd77b8f9ee8f18b0e09175e3f29667e3f265d62e3e02387c34341a4dd3102d60d266b8db24a84e6ae4880462bbe0b557becb7c7642241034308f9e77774f68326fe04ca8cc23283d159eb8b4c119dfa0d6f2e65b6fb37f7894a81b2b8a8b38138c1e7e7c86e087c446899386e9c111e1e418d72437e8762a92e2ae480eb02b01b283a752e075a00d0b7ef077af329b1cb35ec1d52dcffa2032180fc0bbec5f01b4e164b24f8c4d66af2495e7d94f514b292bb67cebfffc05afbef45784fa98457dcc5fbff40a3fbabdcbac2d4592e5592d8713c7b53303deb7dd47ad65ffe088f76edc4a0151728c10eb9c243a4f6bd04c17d48753e222d0c6905cc739a82ae80f60d847d70630ac90de009fbba85d55b90c954918b3e994b7de7e07b1c8997ee0a72e9e657d189170888629568d3175188183c93e21d6999f487f8b9a7ba879831203d3833dfee03ffd47fefeb5bf4124f0adbffc2bceda01d3fa3c9b83c22d04221ea8184bcd5317d7f9bbfbfbcc63e0e6eddb69aac7a5ab46d194e20155c3c28266728c3611931e0c0788f7f4aa1eea3dea35770b8d682ddaf6f14b9790cc2db8546e0398717870c0ed3b3b5416787ccbf1d8198f773527b3bbd8d14d64b84db01e987070708059c46248638aba2256cb893c82c4863befbecb9fbcf706484bb41e5a458ea60d970682906072c801db13b9bcd5a3272d8ec0cebd7b846868956dd87c0a711a701288cd0ce7a0371aa0d5088629084b1eac4d846ec024934aaec2535ae65967928dcc4cb068dcb9b3c383dd03ced0f2f4e5336c550daa110d47dcfdfe5f72ed13ebb8c139e67560efee7d624cbd8fd2779452f9a5bc94306b94cc1164a64095456b1ccc66a49e93e5fe674424c1e3edf1808106bc351c1f1f76d334a911bdec64e2043ff0b8518f9e1f801b820b7957ae1b034815aa652c1253cb8f42b8e0592d4a0cc7ad5b7768ea96b3eb232e9d59a7e76608b0e10293775ee26673cca50f7d8a3b0f02777efc1e160644da3c2e902c43256466dbe50ebb5bf287a218152d300b2db84407208ac614600d656de81978c53581b6a9cb56c00c97a60a08a278d743fb037430c82e6ba8c44e09919495b004b952da8ca5e8caa64c82d9899d8636b4dcbdb7430c3517d62bb647153d5a90d4bda86858dcf821fff0ee9b7ce7dd0977dfdb210d87166c922c2be6468c64bacf68b3262495c8da1225d2c6d851fc6a49119134f0d1f3d0f740e3f1cea7d62379364322693ad863b1c2e9105755584c813c09a0903b65a2b3cc60241ff0e54b562ab332bfd0b4350ff6f6501a2e6eae3174010d99af886981238cbe46d6a6c7583dc7ac774a08ddc0591e411629d3354b582f341d1a8cc1706e75742ca7544d80d370f4fb03caa02aa6994b89a90dd8b610bbde56370c27052664e6bd1492b2e4e7422229ba36d9b22cafdb8693c9849e04ce8c7b48699264f3328b05e0b231f03842e78369230eebe69bf27fadcc69afb8a039c42295fa64b6ddb872e82a6de881385061bcb989e5194bc527be440da325b473082d6aa5fb951d425ae8468cac1bd72a137d3e4da7d8124ae410661669eb86e9c984be46d6078a66535c0daca99c35b64643aa325a48e2092d33cf65a44dbaf3ba0a277d16c50b0caa0ad1d467a543a7118d4ab02a753155b974e9521e64c9f5b815aaa021d6c7a835c912bb31bd454ab704961bcdae914b73ed40142066cb393a33dabaa19e2de83b6558696ae125319625e43d2a6bc33efd6a850eebf46eddb6c556f3d20a672489ba1b549e329a94cc3a0f2909346daa277a95e791ab5750253f29d05d04c29c76f2000975128ee538b4e2f6297d86cefdcb175a022451326bbda4af4208b44d8b1357488baea82f1f8d14b1c7c38acda1c7659f2c3c4019554d7e5926b6b31b92fb11b4acf5613c70745f751decc431eecf6a66ad71f6dc392e9dbf0856f8c6d09de3638d2c8e11ab13474169fce651c934b6b31442c74ea69e5a325f3c96d3679222b421d0849610851053ebadf4380a5761622081b5be7169a362409ea7c265ee300f86e54c726a5854f2108ad46c6f7836d77aa81673cfe48b092d9e7b0753664179f6d9e718af6fa6e89f3380194834b49ec3ec08b145a6170db108310bc14a22e8f69f9722a9d630d3ce8ccaa871cc4321aed7a3371cd39803a9ba9be7f3f3bf1a06bae0a9abdb8ca4c561a0e5110710c96382a74692a0c078a5e5cac54dfa151805e065d205a511cfedbd13a41af2c20b3f43bf3fcadf5b9657225c9a9323b49d8105023177e2cbc8511e47ece6abe8fa1a229a9065114c3a29d506268920196d6c73e9f27926e13e41024a93fa0462684c9c0302de224f9e5fe7dcb0c7e1a4b0a365a0a3585a676fb9508a2881a1339ebd788e0a218ac3659f4e43b0c63444dede5f70e191e778ffd34fe29c746dc898e7b97a61c26cba8350671c92672df23d95941963b6c4d2b64c7550440bf04d1ace8d1d139c089513c6c3218f5c7b923b7b0b4c2a3436282dd69d9726e01cc6a575c74f5d5aa7a7c9d1fa566625f348488a96695902d125815ddee8f1c4f67a4abfa4b641d0641d81c0adfd29fb6dc5473ffe73ac6f8ce99e2acac3a2c488cd0e09b3033a0c99c172142db453b2b63c8290dc2489c8ac2d31a2988ecbf130e22c696a6b5871f9ec655ad964def471e25287480553413435f04560cdb77ce8fa3667aa4065119fb943e9b41f71449c80580f8d1503edf1e16b67383b8c5434385a82589ad520124df8c7db87f4b62ef0d1175fc4b96a05282526ba0a0df5c93eb47512402c53a06d1a9ba68c26e66183982882f2f4595a9baca6334d4d1c524a1b0d7aac0d7a0c7a6b6c5e7a82bbc742941ea639856a7a484c3569a1672d1fb8b4c6739707acb900ea514dda472a9c391c2d427a366480f1c858f8d893e759d3451afe20cd6e62a9df7a7f6abc79bfe6f9177f8e0b972ee4c8a2587e3a4709b87a4a7dfc008d75ee65b854055b9b2c5ccbf0497217cdcf9d25274d0df0d4198b509a1d99c027e270bd21fdd13a7bb305979ff9696e1c196db58688a052507af6334df5c466d5f2b9e7af737dcbe8bb886a1aef29e80081a8827381b3bd86cf3e779947363d6a35657e5a2522d6d04a8fefdf98d03bf7383fff99cf51394d0a903c5a44c48586e668071647686c56d2259461788b2526a415c792c6c50802119723dda97c913f89477b43d636cf72ef64ca85a79ee5d06fb0577b508fcbc70675442ac8810d893cbae9f9d50f3fc6f571cbd837f909991413a21be0d4b3eee6bcf0d83a2f3eb6c5501a905c1865e81ea4c79d69c5eb7b8e4f7cee0b5cb8703911f83953298ac60699efb338b88d8f8b14e94a4a2df808c365f7ec8083e5232c223175fd538347f3045c9e8950d264893ac799f3e7d83dde6378668b479ffb38dfbf35a196c44a25cc93fc2dd508094e0fb5e5f9abebfcc6c71ee743178473bd96a1b6f4b5614de65cac8ef8e863ca675e38cfe66086cf68354aee5588e7c0d6f9eb378eb8f2cca7f8f0473f85f77d96d3e109d657ed827aef06d2eca356af6093822a97cd6cc9459f5962c9a469694f26d487472c0e0e334395419258cc5c6182565122972e9e65f1bdef51cf8ef9d0273ec3577ef037dc383ae2da78442501b5169f3721a42e92d132d4860f5de973e5ec13bcbdd372e3c1098bb6656b245c3f5771fde290715f12c55f1e42318710a9c5f3da7bc74cc78ff16b5ff82d06eb5ba9472a46a0c188b818a98ff78827f79038010b88ae3cdd21a9902bc3e909ac0a1622a16e08d319ed7c41a81b0c97f908a3c3f891340728f979cf4b97cfd1d3c0e1ee3dae7ff0399efdd41778f96b7fc8dae3eb5c744729959acfd5684b79a8c548734fe77ac6d947fb7cf47d632211959a1e759a848b4ab01e268bac4d58b821fff7de9cb767437ee95ffc36172e5dc9a33f104452600b73a88f583cb84daf49a30011e83091159e21d18ea6a4d1a5d0329f4c981f1f42dba2211d1f18524660287c5a92497e724f94eded339cdfde66e7ee5d88c6b3cf7f9c8d273fc677ef35ecea1a4dcf21be45b4066f4467448d440d04675029956be8db94814ce8cb3c3da31133b0d104b38d484b8f1fdc9cf2f7bb3d7ef137fe358f3dfd7e5c65f901db1693f4104cd5ce99df7f079bdd4f96609ef4a873227412d5470a84e2c1140981fae484f9d103acad0966b4aea2e9adc1d6855c86afc0d0e5a055aa6d07fd218f3e7a997bf76ed38486c1da1a9ffc27bfc69ffdc9846fdffe019f7c7483cbfd19ae6db35f965f1568508b6968ac3c23210e2b6c9596c9954074cab419f0ea3b47dcd38b7cf69fff2b1e7bff734805a6a5748e688cf8c58c7af716f1e8163d3b49582056088ea0e4fa2605470da991635169172df3e91c8b1ea97ab8fe26bdcdf30cc6e718f487c8f1ce1bd661ff0c4b4b3562963476fbf6bb7cedebdfe2f39fff7536d6d689067b070ff8f69ffd31876f7e87172e3baeaf7b467181b673d09046092d375e6490a1f8b23417e730f5d4d171f3b8e61f771d83473fc2f39ffe65b62f5e415c7e58c545a2a66e7caf99d1ecbe47bdfb0e557b8832c708681c26f8ac098b42c44770d1b01099b69ee9a245aa3ec3f116d5701df56902a7593484a3bbc8f1ceeb2be558ee77ae700586116dce83fd3dc6e32d067e987f9842994e27fcf0fb2ff1c397fe94cde601cf6e29d7d65afa4c401a88016780a5e175cb1c5a743de632627726fcf8fe94d9e8228f3dff691e7df6230cd6877829b4aa0769886254cd82b07783fac15bb8c5212eb6a4f19f06642d27fd00163282f644fa09d4f536d15e1f570d689a9ac564c2e2e00161728ccd27b878821cddfbc7c4d174bce569d20432a6cf0594517e550022a98b746fef1e6fbcfa12bbfff00aa3e91d1edd84ed5164ad22f53e6345306566ca511b39a81dfbed90c1d9eb5c7bfa792ebcefa994195c059a42b6cb002f48c4b7339a839b34bbefe0170768cc41392b4aa54719a08f28e607486f841f6da1be4f9c07e6c77b348777a90fef61b3093eb479b022c54739baf7232bc4c50a5371ea65e672d11b73cf2243526b09968058680207bbfbdc7ef72d766ffe88f668070913686bbc18daebc370836afd1c5b171ee1c295c7d83a7b11df1b622e3d4c0f89845133bc05849a364c080fdea30bde9902000002b0494441543eb849d54c704dae1f243db41232d214df47fd3ada1fe37b239c2a753de1707787c5ee2e717a886b8e71ed046f6dde9727e289ce21c7f7deb29fb482551a0bc412af1053333305c1989e916833dc2626686e26c436129b9ad04c89618ab880f315eafb783fc4b9fc9c299adb75a5f8c95ca229841699ef51efbd899edc40da2681bdee817f47eba0ad14375cc3f7d7713242f1d87cc1c9ee2da67b7788d37d5c2ba8b5600bcc9a8c32c98c9543d46516bbf488bbe73b5703a766b2a4fc2442264e54b2a568ca3a92f94811c47b7c95881c18778c60378741a1d84be55afc3a20d662614e3c39a0def931d56c0fb505e50194e01cadf7985fc38dd6a9fa43c4226d5dd32ef66176c4627f97fa681fea1a676d6a1499e480dd4f455dd993b4802f3f0724c507f2827331bf846969e12b56b2ca872e0f2c34f80a712b40fea522cae70ce5cbb969a237621208f323ea837bb4c70f907a4aa31551d751578156486f44af3fc2f93e3144ead994d09ce09a29ba38a43dda234e8ed0362410659a719151469845323f9bb912009fb8fe15b728e06a65b0ac739d6e388c87be5f9d752e3fab124f9fd3711e792ecbca310ea397ac2436b4758b8867303e8b6c5e46aa3e55a50983b88a6002a1667eb24f7bb04b2f2ed273a3b323647640383922d635216a4adf44ca739f9239502d849215e51688fd50966095e0cc9b4bafe236ab5650f6691d73b4bc9eac1c97056109ecac9e1d340b5606546b1791f176fec58a0cc0ac21c440d3b6841850a7c8da167def692727b4fbbb300799b51052fc52023137a334077493f494e0725c24359c5dd4321ff1d0be7f2270467ef2a0558bf8ffecfbd41fd3bb7005dddfb3f52935568e7155227d2de24d084da4691b10a85c8f5e3f1300832d6c6854e3487fbccf6cf706f56e1f89b790b88f863a93d03e0b22d71b9239d9ee6158a57515ff0f0c8804d996ada27b0000000049454e44ae426082

hex 也是可以转图片的：

赏

感谢您的支持!

支付宝
微信

本文由 董可伦 发表于 伦少的博客 ,采用署名-非商业性使用-禁止演绎 3.0进行许可。

非商业转载请注明作者及出处。商业转载请联系作者本人。

本文标题：Flink CDC2Kafka 总结

本文链接：https://dongkelun.com/2026/04/22/flink/flinkCDC2Kafka/

欢迎关注我的公众号

扫一扫，分享到微信

微信分享二维码

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true

2016年本科毕业于合肥工业大学<br><br>信息与计算科学专业<br><br>目前在济南一家公司从事大数据的工作~~