Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
463cf8a
[Feature][Spark] Support Spark 3.5.8
goutamadwant Jul 3, 2026
73f43ab
[Feature][E2E] Align Spark runtime with 3.5.8
goutamadwant Jul 4, 2026
467c888
Merge branch 'dev' into GH-11128
goutamadwant Jul 4, 2026
115ecdc
Merge branch 'dev' into GH-11128
goutamadwant Jul 5, 2026
06dcac9
Merge branch 'dev' into GH-11128
goutamadwant Jul 7, 2026
b67cdc8
Merge branch 'dev' into GH-11128
goutamadwant Jul 8, 2026
3780c41
Merge branch 'dev' into GH-11128
goutamadwant Jul 10, 2026
03a9974
Merge branch 'dev' into GH-11128
goutamadwant Jul 11, 2026
41f1c19
Merge branch 'dev' into GH-11128
goutamadwant Jul 12, 2026
7f4edb1
[Feature][Spark] Keep row encoder compatible with Spark 3
goutamadwant Jul 12, 2026
159d406
Merge branch 'dev' into GH-11128
goutamadwant Jul 14, 2026
c3d211a
Merge branch 'dev' into GH-11128
goutamadwant Jul 15, 2026
1ae8cf1
Merge branch 'dev' into GH-11128
goutamadwant Jul 15, 2026
81dda82
[Feature][Spark] Add separate Spark 3.5 runtime
goutamadwant Jul 15, 2026
fbe7137
Merge branch 'dev' into GH-11128
goutamadwant Jul 15, 2026
98c9c7e
Merge remote-tracking branch 'origin/dev' into GH-11128
goutamadwant Jul 16, 2026
1248385
Merge branch 'dev' into GH-11128
goutamadwant Jul 20, 2026
5a5d35d
Merge remote-tracking branch 'origin/dev' into GH-11128
goutamadwant Jul 22, 2026
1591658
[Fix][Test] Fix imports after dev sync
goutamadwant Jul 22, 2026
a861f96
[Fix][Spark] Use Spark 3.5 starter jar for submission
goutamadwant Jul 22, 2026
d3cdf52
Merge remote-tracking branch 'origin/dev' into GH-11128
goutamadwant Jul 22, 2026
08658ff
Merge remote-tracking branch 'origin/dev' into GH-11128
goutamadwant Jul 24, 2026
3683713
[Fix][Spark] Restore batch write commit coordination
goutamadwant Jul 24, 2026
2a36497
Merge branch 'dev' into GH-11128
goutamadwant Jul 25, 2026
b936392
[Fix][Spark] Complete Spark 3.5 connector compatibility
goutamadwant Jul 26, 2026
3724815
[Fix][Spark] Use Spark provided Scala runtime
goutamadwant Jul 26, 2026
cd86d4b
Merge branch 'dev' into GH-11128
goutamadwant Jul 26, 2026
c1057be
[Fix][Spark] Exclude signatures from shaded starter
goutamadwant Jul 26, 2026
1ec0f68
[Fix][Spark] Correct Spark 3.5 launcher defaults
goutamadwant Jul 28, 2026
b574297
Merge branch 'dev' into GH-11128
goutamadwant Jul 28, 2026
da5c4c0
Merge branch 'dev' into GH-11128
goutamadwant Jul 29, 2026
d951008
Merge branch 'dev' into GH-11128
goutamadwant Jul 29, 2026
8a024ef
Merge remote-tracking branch 'origin/dev' into GH-11128
goutamadwant Jul 31, 2026
7dcefcd
[Fix][Spark] Align Iceberg S3 dependencies with Spark 3.5
goutamadwant Jul 31, 2026
99a87d7
Merge branch 'dev' into GH-11128
goutamadwant Aug 1, 2026
123ce82
Merge branch 'dev' into GH-11128
goutamadwant Aug 2, 2026
0ff8b4d
Merge branch 'dev' into GH-11128
goutamadwant Aug 2, 2026
b053226
Merge branch 'dev' into GH-11128
goutamadwant Aug 8, 2026
afa59c6
Merge branch 'dev' into GH-11128
goutamadwant Aug 9, 2026
3704b33
Merge branch 'dev' into GH-11128
goutamadwant Aug 11, 2026
b2cd470
[Improve][Spark] Document cross-version compatibility
goutamadwant Aug 11, 2026
a87908b
Merge remote-tracking branch 'upstream/dev' into audit/11307
goutamadwant Aug 15, 2026
8b99594
Merge remote-tracking branch 'upstream/dev' into audit/11307
goutamadwant Aug 18, 2026
3c13607
Merge remote-tracking branch 'upstream/dev' into GH-11128
goutamadwant Aug 19, 2026
c265f22
Merge remote-tracking branch 'origin/GH-11128' into GH-11128
goutamadwant Aug 19, 2026
060808d
[Fix][Spark] Stage compatible S3 dependencies for Spark 3.5
goutamadwant Aug 20, 2026
4de479e
[Fix][Spark] Isolate Hudi S3 tests by engine
goutamadwant Aug 22, 2026
e4e6ec7
[Fix][Spark] Pass bucket to SeaTunnel Hudi S3 test
goutamadwant Aug 26, 2026
169864a
[Fix][E2E] Preserve Druid decimal precision on Spark 3.5
goutamadwant Sep 3, 2026
0f2c42c
[Docs][Spark] Clarify existing Spark 3.4 starter guidance
goutamadwant Sep 12, 2026
ee21221
[Fix][Core] Preserve Spark 3.5 launcher arguments and exit codes
goutamadwant Sep 13, 2026
2b832dc
[Fix][E2E] Preserve Spark 3.5 staging with upstream MinIO mirror
goutamadwant Sep 13, 2026
dd36ee0
Merge branch 'dev' into GH-11128
goutamadwant Sep 15, 2026
d3d34f2
[Test][Spark] Accept Spark 3.5 in Python transform container policy
goutamadwant Sep 15, 2026
629432c
[Docs][Spark] Clarify Windows launcher argument limitations
goutamadwant Sep 19, 2026
b1484e5
[Fix][Spark] Document launcher protocol and verify packaged usage
goutamadwant Sep 19, 2026
457a0d1
[Test][Spark] Verify packaged launcher on native Windows
goutamadwant Sep 21, 2026
fcc25db
[Fix][Spark] Preserve quoted Windows launcher arguments and isolate n…
goutamadwant Sep 22, 2026
33e30bb
[Chore] Merge latest dev for upstream CI fixes
goutamadwant Sep 22, 2026
70f4b60
Merge remote-tracking branch 'upstream/dev' into GH-11128
goutamadwant Sep 28, 2026
b1718c2
[Feature][Spark] Run the Spark 3.5 E2E container outside PR runs
goutamadwant Sep 28, 2026
c8b60d5
Merge remote-tracking branch 'upstream/dev' into GH-11128
goutamadwant Oct 1, 2026
d55343e
[Chore][Spark] Sync 3.5.8 support with dev CI updates
goutamadwant Oct 10, 2026
cadcf87
Merge remote-tracking branch 'upstream/dev' into GH-11128
goutamadwant Oct 17, 2026
5c18547
[Test][Spark] Verify isolated launcher logging and clarify supported …
goutamadwant Oct 24, 2026
d804587
[Test][Spark] Validate micro-batch sink and packaged Unix launcher
goutamadwant Oct 24, 2026
28505d7
Merge branch 'dev' into GH-11128
goutamadwant Oct 24, 2026
380764c
[Test][Spark] Run packaged Spark 3.5 launcher checks on Unix
goutamadwant Oct 25, 2026
4b80eba
[Test][Core] Guard Spark file-stream test on Windows
goutamadwant Oct 28, 2026
5184b37
[Fix][CI] Isolate packaged Spark launcher integration tests
goutamadwant Oct 30, 2026
47ae1be
[Docs][Spark] Clarify validation scope and merge dev checkpoint fix
goutamadwant Nov 1, 2026
1822620
[Test][Spark] Isolate starter properties and Spark contexts
goutamadwant Nov 3, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/backend.yml
Original file line number Diff line number Diff line change
Expand Up @@ -712,6 +712,10 @@ jobs:
# flags must be repeated here as well as in the pom.xml surefire.module.args property.
SUREFIRE_JVM_ARGS: -Xmx3072m -XX:MaxMetaspaceSize=512m -Dfile.encoding=UTF-8 --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-exports=java.security.jgss/sun.security.krb5=ALL-UNNAMED
if: runner.os == 'Windows'
- name: Run packaged Spark 3.5 launcher tests
run: ./mvnw -B -T 1 verify -pl seatunnel-core/seatunnel-spark-starter/seatunnel-spark-3.5-starter -am -DskipUT=true -DskipIT=false -D"it.test=TestSpark35StarterJarContentIT" -D"it.failIfNoSpecifiedTests=false" -D"license.skipAddThirdParty"=true -D"skip.ui"=true --no-snapshot-updates
env:
MAVEN_OPTS: -Xmx1536m -Dfile.encoding=UTF-8
- name: Upload test reports
if: failure()
uses: ./.github/actions/upload-test-reports
Expand Down
2 changes: 2 additions & 0 deletions docs/en/engines/command/usage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ bin/start-seatunnel-spark-2-connector-v2.sh
bin/start-seatunnel-spark-3-connector-v2.sh
```

For Spark 3.5.x, use `bin/start-seatunnel-spark-3.5-connector-v2.sh` with the same options. Set `SPARK_HOME` to the matching Spark installation. See [Spark engine compatibility](../spark.md) for the current validation limits.

</TabItem>
<TabItem value="flink13">

Expand Down
42 changes: 42 additions & 0 deletions docs/en/engines/spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,46 @@ Use this path if you want to run SeaTunnel on Spark:
- [Quick Start With Spark](../getting-started/locally/quick-start-spark.md)
- [Job Configuration Guide](../getting-started/job-configuration-guide.md)

SeaTunnel provides separate Spark 3 starters so existing Spark 3.3 deployments can remain on
their current runtime while Spark 3.5 deployments use the Spark 3.5.8 starter:

- `seatunnel-spark-3-starter.jar` for Spark 3.3.x
- `seatunnel-spark-3.5-starter.jar` for Spark 3.5.x

For an existing Spark 3.4 deployment, retain the existing `seatunnel-spark-3-starter.jar`
and `start-seatunnel-spark-3-connector-v2` launcher. This change does not add a separate
Spark 3.4 starter or extend its compatibility coverage.

The Spark 3.5 starter reuses the translation layer compiled against Spark 3.3.
Row-encoder, batch-write and bounded micro-batch sink tests exercise selected paths on Spark 3.5.8, but do not
prove compatibility of every Catalyst or DataSource V2 path. Other binary
incompatibilities may surface only when those paths execute on the target runtime.
Validate your connectors and deployment mode before upgrading.

The public Spark 3.5 starter uses the batch `read()` and `write().save()` pipeline;
use a batch job configuration with this launcher. The micro-batch test runs a finite
Spark file stream through the row encoder and SeaTunnel streaming sink adapter. It
does not validate the SeaTunnel streaming source, checkpoint restore, or streaming
jobs submitted through the public starter. Spark 3.5 connector-container tests are
scheduled-only (`testInPR=false`); normal PR CI does not run that runtime matrix.

The Spark 3.5 Windows `.cmd` launcher still passes a command string to `spark-submit.cmd`;
it does not use the Unix launcher's argument-file protocol. Arbitrary arguments
containing embedded quotes or Windows command-shell metacharacters are not
supported, and native Windows launcher behavior has not been validated. Do not
pass untrusted values through this launcher; use the Unix launcher when exact
argument preservation is required.

The packaged Windows tests exercise help, JVM startup failure, and opening quoted
configuration paths containing spaces, `!`, and `&`. The path tests stop at a
configuration parse error, before Spark submission; they do not establish exact
argument preservation by `spark-submit.cmd`. Native Windows CI must pass on the
revision being validated before treating these launcher cases as verified.

The Unix packaged test runs the real starter JVM and checks argument preservation
for paths containing spaces and both quote characters. It captures the arguments
at `spark-submit`; it does not submit a Spark job.

## When To Choose Spark

Spark is usually the right engine when:
Expand Down Expand Up @@ -49,6 +89,8 @@ Spark on YARN client mode:
./bin/start-seatunnel-spark-3-connector-v2.sh --master yarn --deploy-mode client --config config/example.conf
```

For Spark 3.5, use `start-seatunnel-spark-3.5-connector-v2.sh` with the same options.

## Minimal Example Job

The example below runs on Spark and prints generated records to the console.
Expand Down
2 changes: 2 additions & 0 deletions docs/en/getting-started/docker/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,8 @@ docker run --rm -it apache/seatunnel bash ./bin/start-seatunnel-spark-2-connecto
docker run --rm -it apache/seatunnel bash ./bin/start-seatunnel-spark-3-connector-v2.sh -c config/v2.batch.config.template
```

For Spark 3.5.x, use `start-seatunnel-spark-3.5-connector-v2.sh` instead. The container must include Spark 3.5.x, the matching SeaTunnel starter, and `SPARK_HOME` pointing to that Spark installation; changing the launcher name alone does not upgrade the image. See [Spark engine compatibility](../../engines/spark.md) for validation limits.

- Flink
before you submit job, you need start flink cluster first.

Expand Down
29 changes: 27 additions & 2 deletions docs/en/getting-started/locally/quick-start-spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,12 @@ see [Getting Started: Standalone](https://spark.apache.org/docs/latest/spark-sta

## Step 3: Add Job Config File To Define A Job

Edit `config/v2.streaming.conf.template`, which determines the way and logic of data input, processing, and output after seatunnel is started.
For Spark 3.5, edit `config/v2.batch.config.template` with `job.mode = "BATCH"`.
The bounded Spark 3.5.8 micro-batch test covers the row encoder and streaming sink
adapter. Streaming through the public starter and checkpoint restore remain
unvalidated, so the streaming template is not a verified Spark 3.5 example. Existing
Spark 2.4/3.3 deployments can continue
using `config/v2.streaming.conf.template` as shown below.
The following is an example of the configuration file, which is the same as the example application mentioned above.

```hocon
Expand Down Expand Up @@ -89,7 +94,7 @@ cd "apache-seatunnel-${version}"
--config ./config/v2.streaming.conf.template
```

Spark3.x.x
Spark 3.3.x

```shell
cd "apache-seatunnel-${version}"
Expand All @@ -99,6 +104,26 @@ cd "apache-seatunnel-${version}"
--config ./config/v2.streaming.conf.template
```

Spark 3.4.x

For an existing Spark 3.4 deployment, retain `seatunnel-spark-3-starter.jar` and the
`start-seatunnel-spark-3-connector-v2` launcher shown above. This change adds no
separate Spark 3.4 starter and no new Spark 3.4 compatibility validation.

Spark 3.5.x

Start with a batch job. The bounded micro-batch sink-adapter test does not validate
streaming through the public starter or checkpoint restore; the streaming template
is not part of this quick start for Spark 3.5.

```shell
cd "apache-seatunnel-${version}"
./bin/start-seatunnel-spark-3.5-connector-v2.sh \
--master local[4] \
--deploy-mode client \
--config ./config/v2.batch.config.template
```

**See The Output**: When you run the command, you can see its output in your console. This
is a sign to determine whether the command ran successfully or not.

Expand Down
2 changes: 2 additions & 0 deletions docs/zh/engines/command/usage.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ bin/start-seatunnel-spark-2-connector-v2.sh
bin/start-seatunnel-spark-3-connector-v2.sh
```

对于 Spark 3.5.x,请使用 `bin/start-seatunnel-spark-3.5-connector-v2.sh`,选项相同。将 `SPARK_HOME` 指向匹配的 Spark 安装目录。当前验证范围请参阅 [Spark 引擎兼容性](../spark.md)。

</TabItem>
<TabItem value="flink13">

Expand Down
35 changes: 35 additions & 0 deletions docs/zh/engines/spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,39 @@
- [Spark 引擎快速开始](../getting-started/locally/quick-start-spark.md)
- [作业配置指南](../getting-started/job-configuration-guide.md)

SeaTunnel 提供独立的 Spark 3 启动器。现有 Spark 3.3 部署可以继续使用原有运行时,
Spark 3.5 部署则可以使用 Spark 3.5.8 启动器:

- `seatunnel-spark-3-starter.jar` 用于 Spark 3.3.x
- `seatunnel-spark-3.5-starter.jar` 用于 Spark 3.5.x

现有 Spark 3.4 部署应继续使用原来的 `seatunnel-spark-3-starter.jar` 和
`start-seatunnel-spark-3-connector-v2` 启动脚本。本次变更不新增独立的 Spark 3.4
启动器,也不扩展其兼容性验证范围。

Spark 3.5 启动器复用了基于 Spark 3.3 编译的转换层。行编码器、批量写入和有界微批写入测试在
Spark 3.5.8 上验证了部分路径,但不能证明所有 Catalyst 或 DataSource V2 路径均兼容。
其他二进制兼容性问题可能仅在目标运行时执行相关路径时暴露。升级前请验证所用连接器
和部署模式。

Spark 3.5 公共启动器使用批处理的 `read()` 和 `write().save()` 执行路径,应使用批处理作业配置。
微批测试将有限的 Spark 文件流通过行编码器写入 SeaTunnel 流式 Sink 适配器,不覆盖 SeaTunnel
流式 Source、检查点恢复,或通过公共启动器提交流处理作业。Spark 3.5 连接器容器测试仅在定时
任务中运行(`testInPR=false`),普通 PR CI 不运行该运行时矩阵。

Spark 3.5 Windows `.cmd` 启动器仍将命令字符串传递给 `spark-submit.cmd`,未采用 Unix
启动器的参数文件协议。不支持包含嵌入引号或 Windows 命令解释器元字符的任意参数,
且尚未验证原生 Windows 启动器行为。请勿通过此启动器传入不可信的值;需要精确保留
参数时,请使用 Unix 启动器。

Windows 打包测试覆盖帮助输出、JVM 启动失败,以及打开包含空格、`!` 和 `&` 的带引号
配置路径。路径测试在配置解析错误处停止,不会提交 Spark 作业,因此不能证明
`spark-submit.cmd` 会精确保留参数。只有待验证版本的原生 Windows CI 通过后,
才能将这些启动器场景视为已验证。

Unix 打包测试运行真实的启动器 JVM,检查包含空格及单、双引号的路径参数是否完整保留。
该测试在 `spark-submit` 边界捕获参数,不会提交 Spark 作业。

## 什么时候选择 Spark

以下场景通常更适合使用 Spark:
Expand Down Expand Up @@ -49,6 +82,8 @@ Spark on YARN 客户端模式:
./bin/start-seatunnel-spark-3-connector-v2.sh --master yarn --deploy-mode client --config config/example.conf
```

使用 Spark 3.5 时,请使用 `start-seatunnel-spark-3.5-connector-v2.sh`,其他参数保持不变。

## 最小示例作业

下面这个例子会在 Spark 上运行,并把生成的数据打印到控制台。
Expand Down
2 changes: 2 additions & 0 deletions docs/zh/getting-started/docker/docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,8 @@ docker run --rm -it apache/seatunnel bash ./bin/start-seatunnel-spark-2-connecto
docker run --rm -it apache/seatunnel bash ./bin/start-seatunnel-spark-3-connector-v2.sh -c config/v2.batch.config.template
```

对于 Spark 3.5.x,请改用 `start-seatunnel-spark-3.5-connector-v2.sh`。容器必须包含 Spark 3.5.x 和匹配的 SeaTunnel 启动器,并将 `SPARK_HOME` 指向该 Spark 安装目录;仅更改启动器名称不会升级镜像。验证范围请参阅 [Spark 引擎兼容性](../../engines/spark.md)。

- Flink
在提交作业之前,您需要先启动 Flink 集群。

Expand Down
26 changes: 24 additions & 2 deletions docs/zh/getting-started/locally/quick-start-spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,10 @@ sidebar_position: 4

## 步骤 3: 添加作业配置文件来定义作业

编辑`config/v2.streaming.conf.template`,它决定了当SeaTunnel启动后数据输入、处理和输出的方式及逻辑。
Spark 3.5 请编辑 `config/v2.batch.config.template`,并设置 `job.mode = "BATCH"`。
Spark 3.5.8 的有界微批测试覆盖行编码器和流式 Sink 适配器。通过公共启动器执行流处理和
检查点恢复仍未验证,因此流处理模板不是已验证的 Spark 3.5 示例。
现有 Spark 2.4/3.3 部署可以继续使用下文所示的 `config/v2.streaming.conf.template`。
下面是配置文件的示例,它与上面提到的示例应用程序相同。

```hocon
Expand Down Expand Up @@ -89,7 +92,7 @@ cd "apache-seatunnel-${version}"
--config ./config/v2.streaming.conf.template
```

Spark 3.x.x
Spark 3.3.x

```shell
cd "apache-seatunnel-${version}"
Expand All @@ -99,6 +102,25 @@ cd "apache-seatunnel-${version}"
--config ./config/v2.streaming.conf.template
```

Spark 3.4.x

现有 Spark 3.4 部署应继续使用 `seatunnel-spark-3-starter.jar` 和上文所示的
`start-seatunnel-spark-3-connector-v2` 启动脚本。本次变更不新增独立的 Spark 3.4
启动器,也不新增 Spark 3.4 兼容性验证。

Spark 3.5.x

请先运行批处理作业。有界微批 Sink 适配器测试不验证通过公共启动器执行流处理或
检查点恢复,因此 Spark 3.5 的快速开始不使用流处理模板。

```shell
cd "apache-seatunnel-${version}"
./bin/start-seatunnel-spark-3.5-connector-v2.sh \
--master local[4] \
--deploy-mode client \
--config ./config/v2.batch.config.template
```

**查看输出**: 当您运行该命令时,您可以在控制台中看到它的输出。您可以认为这是命令运行成功或失败的标志。

SeaTunnel控制台将会打印一些如下日志信息:
Expand Down
1 change: 1 addition & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@
<flink.1.20.1.version>1.20.1</flink.1.20.1.version>
<spark.2.4.0.version>2.4.0</spark.2.4.0.version>
<spark.3.3.0.version>3.3.0</spark.3.3.0.version>
<spark.3.5.8.version>3.5.8</spark.3.5.8.version>
<spark.binary.2.4.version>2.4</spark.binary.2.4.version>
<commons.beanutils.version>1.9.4</commons.beanutils.version>
<commons.cli.version>1.4</commons.cli.version>
Expand Down
12 changes: 11 additions & 1 deletion seatunnel-core/seatunnel-spark-starter/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
<modules>
<module>seatunnel-spark-2-starter</module>
<module>seatunnel-spark-3-starter</module>
<module>seatunnel-spark-3.5-starter</module>
<module>seatunnel-spark-starter-common</module>
</modules>

Expand Down Expand Up @@ -71,11 +72,12 @@
jul-to-slf4j
jcl-over-slf4j

Spark(3.x) server lib already include:
Spark(3.x) server lib already include (binding varies by version):
slf4j-api
log4j-api
log4j-core
log4j-slf4j-impl
log4j-slf4j2-impl
log4j-1.2-api
jul-to-slf4j
jcl-over-slf4j
Expand All @@ -96,7 +98,15 @@
<exclude>ch.qos.logback:*</exclude>
<exclude>org.apache.logging.log4j:log4j-api</exclude>
<exclude>org.apache.logging.log4j:log4j-core</exclude>
<!--
These are shade exclusions, not starter/logging includes.
The launcher JVM uses SeaTunnel's SLF4J 1.7 API and
log4j-slf4j-impl from starter/logging. Spark submission
starts a separate JVM using Spark's own logging libraries;
neither binding should be embedded in the starter jar.
-->
<exclude>org.apache.logging.log4j:log4j-slf4j-impl</exclude>
<exclude>org.apache.logging.log4j:log4j-slf4j2-impl</exclude>
<!-- spark3.x use slf4j + log4j2.x -->
<exclude>org.apache.logging.log4j:log4j-to-slf4j</exclude>
<exclude>org.apache.seatunnel:seatunnel-shade-hadoop3-uber</exclude>
Expand Down
Loading
Loading