暂无图片
暂无图片
暂无图片
暂无图片
暂无图片

2025年 Apache SeaTunnel 2月份社区月报速递

SeaTunnel 2025-03-07
40

SeaTunnel 社区月报

2025-02期

“ 各位热爱 Apache SeaTunnel 的小伙伴们,SeaTunnel 社区月报来啦!”

SeaTunnel 正在迅猛发展,这将有利于提升数据同步的高可扩展性、高性能及高可靠性。

SeaTunnel Monthly Merge Stars

感谢以下小伙伴上个月为 Apache SeaTunnel 做的精彩贡献(排名不分先后):

@hawk9821@fcb-xiaobo@akulabs8
@realcpf@lyne7-sc@litiliu
@uniding@mickeyzzm@MyeoungDev
@thakralgarvit@xiaochen-zhou@zhangshenghang
@remones@Gemini147258@Scorpio777888
@lm-ylj@joyCurry30@hailin0
@CosmosNi@duzhendi@jw-itq
@wuchunfu@liunaijie@corgy-w
@wattt3@Hisoka-X@SplitfireUptown
@quicklyfast@wgzhao@liugddx

近期修复和改进

修复内容

  • [Fix][Connector-File]解决冲突的`file_format_type`需求 [1] @litiliu
  • [Fix]修复SourceSplitEnumerator实现类中的错误日志名称 [2] @fcb-xiaobo
  • [Fix][Doc]修复S3文件文档 [3] @litiliu
  • [Fix][Connector-V2]修复解析SqlServer JDBC URL错误 [4] @Hisoka-X
  • [Fix][Connector-V2]修复连接器中CUSTOM_SQL的描述 [5] @xiaochen-zhou
  • [Bugfix]修复ExceptionUtil的ClassCastException [6] @hailin0
  • [Fix][E2e]优化后的POM文件名称标签 [7] @corgy-w
  • [Fix][Connector-V2]修复在 request_tablet_size 小于 BUCKETS 数量的场景中可能的数据丢失问题 [8] @xiaochen-zhou
  • [Fix][Connector-v2]将DateMilliConvertor添加到将DateMilliVector转换为默认时区 [9] @xiaochen-zhou
  • [Fix][Transform]修复FieldMapper转换时丢失字段约束信息 [10] @quicklyfast
  • [Bugfix][Canal]修复运河序列化为json [11] @hailin0
  • [Fix][Connector-V2]修复JDBC Sink语句缓冲区错误的时间清除问题 [12] @Hisoka-X
  • [Fix][Connector-V2]修复无法读取空字符串的文件阅读问题 [13] @corgy-w
  • [Fix][transform-v2]大型语言模型转换提供者切换案例缺少中断 [14] @realcpf
  • [Fix][Connector-v2][DorisIT]修复DorisIT无法运行本地的问题 [15] @zhangshenghang
  • [Fix][transform-v2]SQL转换支持最大/最小函数 [16] @CosmosNi
  • [hotfix][doc]HdfsFile 文档 [17] @wgzhao
  • [Fix][Plugin]优化插件发现机制 [18] @corgy-w
  • [Fix][MySQL-CDC]修复因binlog删除导致的恢复任务失败 [19] @lm-ylj

improve

  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Vertica.md]系统 [20] @Scorpio777888
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/SqlServer.md]系统 [21] @Scorpio777888
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/SelectDB-Cloud.md]系统 [22] @Scorpio777888
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/S3File.md]系统 [23] @Scorpio777888
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/FtpFile.md]系统 [24] @Scorpio777888
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/FakeSource.md]系统 [25] @Scorpio777888
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Snowflake.md]系统 [26] @Scorpio777888
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/SftpFile.md]系统 [27] @Scorpio777888
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/Easysearch.md]系统 [28] @Scorpio777888
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/DB2.md]系统 [29] @Scorpio777888
  • [Improve][Redis]优化的Redis连接参数 [30] @fcb-xiaobo
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/S3-Redshift.md]系统 [31] @Gemini147258
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/RocketMQ.md]系统 [32] @Gemini147258
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/OssJindoFile.md]系统 [33] @Gemini147258
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/OssFile.md]系统 [34] @Gemini147258
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Oracle.md]系统 [35] @Gemini147258
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/OceanBase.md]系统 [36] @Gemini147258
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/ObsFile.md]系统 [37] @Gemini147258
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/CosFile.md]系统 [38] @Gemini147258
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/Clickhouse.md]系统 [39] @Gemini147258
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Mysql.md]系统 [40] @Gemini147258
  • [Improve][Jdbc]删除无用的工具。[41] @joyCurry30
  • [Feature][Transform-v2]在嵌入模块和LLM模块中添加对智谱AI的支持 [42] @xiaochen-zhou
  • [Feature][Connector-V2]为读写文件功能添加`filename_extension`参数。[43] @Hisoka-X
  • [Improve][Zeta]默认禁用RESTful API v1 [44] @Hisoka-X
  • [improve]更新ClickHouse连接器配置选项 [45] @lyne7-sc
  • [Improve][connector][activemq]移除重复的依赖项 [46] @wuchunfu
  • [Improve][Connector-V2]改进兽人阅读错误信息 [47] @Hisoka-X
  • [improve]德瑞选项 [48] @liunaijie
  • [improve]数据枢纽接收选项 [49] @liunaijie
  • [improve]控制台接收选项 [50] @liunaijie
  • [improve]钉钉接收器选项 [51] @liunaijie
  • [improve]重新组织Zeta引擎配置选项 [52] @liunaijie
  • [improve]RabbitMQ 选项 [53] @liunaijie
  • [Feature][Config]支持自定义配置键用于加密/解密 [54] @remones
  • [improve]Slack连接器选项 [55] @akulabs8
  • [Improve][Connector-V2]为RocketMQ连接器添加可选标志,以跳过解析错误而不是失败 [56] @xiaochen-zhou
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/TDengine.md]系统 [57] @duzhendi
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Tablestore.md]系统 [58] @duzhendi
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Socket.md]系统 [59] @duzhendi
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Milvus.md]系统 [60] @duzhendi
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Kudu.md]系统 [61] @duzhendi
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/IoTDB.md]系统 [62] @duzhendi
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/Phoenix.md]系统 [63] @mickeyzzm
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/OpenMldb.md]系统 [64] @mickeyzzm
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/Maxcompute.md]系统 [65] @mickeyzzm
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/GoogleSheets.md]系统 [66] @mickeyzzm
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/Cassandra.md]系统 [67] @mickeyzzm
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/AmazonSqs.md]系统 [68] @mickeyzzm
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Slack.md]系统 [69] @mickeyzzm
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/CosFile.md]系统 [70] @mickeyzzm
  • [Doc][Improve] support chinese [docs/zh/connector-v2/source/AmazonDynamoDB.md]系统 [71] @mickeyzzm
  • [Improve][Doc]更新文档中的旧链接 [72] @thakralgarvit
  • [Doc][Improve] support chinese [docs/zh/connector-v2/sink/Sentry.md]#8497 [73] @mickeyzzm
  • [improve]更新Kafka源默认模式从 content<row> 到 content</row[74] @liunaijie
  • [Improve][dist]减少Docker镜像的大小 [75] @quicklyfast
  • [Improve][Connector-v2][Paimon]派蒙目录关闭错误信息更新 [76] @uniding
  • [improve]添加StarRocks选项 [77] @fcb-xiaobo
  • [Improve]重构连接器通用选项 [78] @liunaijie
  • [improve]更新Redis连接器配置选项 [79] @akulabs8
  • [Improve][e2e]移除重复的依赖项 [80] @wuchunfu
  • [Improve][Jdbc]支持在OpenGauss中使用upsert [81] @hailin0
  • [Improve][Jdbc]改进目录连接缓存 [82] @hailin0
  • [improve]添加Elasticsearch选项 [83] @fcb-xiaobo
  • [Improve][Zeta]作业主中的类加载器拆分 [84] @Hisoka-X
  • [improve]添加断言选项 [85] @liunaijie
  • [improve]Kafka连接器选项 [86] @wattt3
  •  [improve]更新S3File连接器配置选项 [87] @litiliu
  • [improve]Cassandra连接器选项 [88] @liunaijie
  • [improve]亚马逊SQS连接器更新 [89] @liunaijie
  • [improve]更新 Amazon DynamoDB 连接器 [90] @liunaijie
  • [improve]添加连接器选项类存在检查 [91] @liunaijie
  • [improve]更新Druid连接器配置选项 [92] @akulabs8
  • [Doc][Improve]翻译与Redis/派蒙相关的中文文档 [93] @lyne7-sc
  • [Feature][Connector-V2]支持多表的MaxCompute源 [94] @Hisoka-X
  • [improve]更新ActiveMQ连接器配置选项 [95] @liunaijie
  • [Feature][transform-v2]JSONPath 支持 Map 数组类型 [96] @CosmosNi

others

  • [Doc][FTP]修复FTP文档中的死链接。[97] @joyCurry30
  • [Doc]添加 Milvus 源中文文档 [98] @mickeyzzm
  • [ [Docs][S3File]"Orc 数据类型" 应在文档中更正为 "Parquet 数据类型" ]( https://github.com/apache/seatunnel/pull/8705 " [Docs][S3File]"Orc 数据类型" 应在文档中更正为 "Parquet 数据类型" ") @MyeoungDev
  • [Refactor][core]统一转换工厂(transformFactory)的创建逻辑 [99] @liugddx

apache/seatunnel

fix

  • [Fix][Connector-V2]用户选择CSV字符串模式 [100] @corgy-w
  • [Fix][mysql-cdc]在启动时修复GTID以正确从检查点恢复 [101] @jw-itq
  • [Fix][connector-http]当帖子包含参数时进行修复 [102] @CosmosNi

improve

  • [Improve][CDC]过滤心跳事件 [103] @hailin0
  • [Feature][Transforms-V2]处理大型语言模型非标准格式响应 [104] @SplitfireUptown
  • [Feature][Jdbc]支持达梦数据库的sink DDL [105] @hawk9821
  • [Feature][Core]增加槽位分配策略 [106] @zhangshenghang

apache/seatunnel-website

improve

  • [Improve][Team]添加名为张东浩的新提交者 [107] @hawk9821
  • [Improve]更新提交代码指南 [108] @Hisoka-X




同步Demo

 MySQL→Doris | MySQLCDC | MySQL→Hive | HTTP → Doris  | HTTP → MySQL |  MySQL→StarRocks | MySQL→Elasticsearch | Kafka→ClickHouse

新手入门

 SeaTunnel 让数据集成变得 So easy!  3 分钟入门指南
 0 到 1 快速入门 / 初探 / 深入理解  
  分布式集群部署 | CDC数据同步管道 | Oracle-CDC

最佳实践

 OPPO | 清风 | 天翼云 | 马蜂窝 | 孩子王 | 哔哩哔哩 | 唯品会 | 众安保险 | 兆原数通 | 亚信科技 | 映客 | 翼康济世 | 信也科技 | 华润置地 | Shopee | 京东科技 | 58同城 | 互联网银行 | JPMorgan

测试报告

SeaTunnel VS GLUE |  VS Airbyte |  VS DataX | SeaTunnel 与 DataX 、Sqoop、Flume、Flink CDC 对比

源码解析

Zeta引擎源码解析(一) |(二) |(三)| API 源码解析 | 2.1.1源码解析 | 封装 Flink 连接数据库解析

 


Apache SeaTunnel





Apache SeaTunnel 是一个分布式、高性能、易扩展、用于海量数据(离线&实时)同步和转化的数据集成平台

仓库地址: 
https://github.com/apache/seatunnel
网址:
https://seatunnel.apache.org/
Apache SeaTunnel 下载地址:
https://seatunnel.apache.org/download
衷心欢迎更多人加入!
我们相信,在「Community Over Code」(社区大于代码)、「Open and Cooperation」(开放协作)、「Meritocracy」(精英管理)、以及「多样性与共识决策」等 The Apache Way 的指引下,我们将迎来更加多元化和包容的社区生态,共建开源精神带来的技术进步!
我们诚邀各位有志于让本土开源立足全球的伙伴加入 SeaTunnel 贡献者大家庭,一起共建开源!
提交问题和建议:
https://github.com/apache/seatunnel/issues
贡献代码:
https://github.com/apache/seatunnel/pulls
订阅社区开发邮件列表 : 
dev-subscribe@seatunnel.apache.org
开发邮件列表:
dev@seatunnel.apache.org
加入 Slack:
https://join.slack.com/t/apacheseatunnel/shared_invite/zt-1kcxzyrxz-lKcF3BAyzHEmpcc4OSaCjQ
关注 Twitter: 
https://twitter.com/ASFSeaTunnel

文章转载自SeaTunnel,如果涉嫌侵权,请发送邮件至:contact@modb.pro进行举报,并提供相关证据,一经查实,墨天轮将立刻删除相关内容。

评论