Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/en/engines/zeta/python-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,16 @@ with SeaTunnelClient(base_url="http://localhost:8080") as client:
print(job_details)
```

An unknown id is still an HTTP 200 response and contains `"found": false`. Use `job_exists`
when you only need an existence check; it also supports older servers that return only `jobId`
for an unknown job:

```python
with SeaTunnelClient(base_url="http://localhost:8080") as client:
if not client.jobs.job_exists(jobId=12345):
print("The job id is unknown or has expired from finished-job history")
```

### Cluster Monitoring

You can use the `cluster` property to get cluster information.
Expand Down
2 changes: 1 addition & 1 deletion docs/en/engines/zeta/rest-api-job-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -389,7 +389,7 @@ curl http://<master>:8080/overview | \
|---|---|---|
| `HTTP 404` on any endpoint | REST API not enabled or wrong port | Set `enable-http: true` and check port |
| `Connection refused` | Master not started or firewall blocking port | Verify master process is running; check firewall |
| `jobId not found` in `job-info` | Job has already finished or was never started | Query `/finished-jobs/:state` with the expected final state |
| `found: false` (or a status-less 200) in `job-info` | The id is unknown, or the job has expired from finished-job history | Query `/finished-jobs/:state` with the expected final state |
| Submit returns `400 Bad Request` | Malformed JSON or missing required fields | Validate JSON; check `plugin_name` spelling |
| `Job already exists with same job.id` | Submitting duplicate `job.id` without stopping first | Cancel or stop the existing job, then resubmit |
| `Unauthorized 401` | Basic authentication is enabled but no credentials were provided | Include `-u user:pass` |
Expand Down
20 changes: 14 additions & 6 deletions docs/en/engines/zeta/rest-api-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -399,7 +399,7 @@ None.
}
```

`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` always be returned.
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` are returned for a job the cluster knows; an unknown id returns the `found: false` response below.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` will return when job is running.
`finishTime`, `errorMsg` will return when job is finished.
A running job also returns a `diagnostics` block (state timestamps and per-pipeline restore counts),
Expand Down Expand Up @@ -427,14 +427,18 @@ not.
| TableSinkWrite* | Per-table sink write attempts, key format `TableSinkWriteXXX#<table>` |
| TableSinkCommitted* | Per-table sink committed metrics, key format `TableSinkCommittedXXX#<table>` |

When we can't get the job info, the response contains only the requested `jobId`:
An unknown id is still an HTTP 200 response with `found: false`:

```json
{
"jobId" : "733584788375666689"
"jobId" : "<id>",
"found" : false
}
```

See the `/job-info/:jobId` section in [REST API V2](rest-api-v2.md) for details and the
older-server fallback.

</details>

------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -513,18 +517,22 @@ This API has been deprecated, please use /hazelcast/rest/maps/job-info/:jobId in
}
```

`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` always be returned.
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` are returned for a job the cluster knows; an unknown id returns the `found: false` response below.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` will return when job is running.
`finishTime`, `errorMsg` will return when job is finished.

When we can't get the job info, the response contains only the requested `jobId`:
An unknown id is still an HTTP 200 response with `found: false`:

```json
{
"jobId" : "733584788375666689"
"jobId" : "<id>",
"found" : false
}
```

See the `/job-info/:jobId` section in [REST API V2](rest-api-v2.md) for details and the
older-server fallback.

</details>

------------------------------------------------------------------------------------------
Expand Down
24 changes: 18 additions & 6 deletions docs/en/engines/zeta/rest-api-v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -674,7 +674,8 @@ This endpoint helps troubleshoot why jobs stay in `PENDING` by showing the pendi
}
```

`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` always be returned.
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` are returned for a job the
cluster knows. An unknown id returns the `found: false` response described below.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` will return when job is running.
`finishTime`, `errorMsg` will return when job is finished.
`diagnostics` will return when the job is running and its diagnostics can be read from the master
Expand Down Expand Up @@ -716,14 +717,21 @@ for every running job would cost one more round trip to the master per job.
| pipelines[].stateTimestamps | Epoch millis when the pipeline entered each state. After a restore, the timestamps of the new attempt overwrite the previous ones |
| totalPipelineRestoreCount | Sum of `restoreCount` over all pipelines of the job |

When we can't get the job info, the response contains only the requested `jobId`:
An unknown id is still an HTTP 200 response. It contains the requested id and an explicit
`found: false` marker:

```json
{
"jobId" : "733584788375666689"
"jobId" : "733584788375093248",
"found" : false
}
```

`found: false` means the cluster has no retained record for the id: the id may be wrong, or the
job may have expired from the finished-job history. Successful payloads keep their existing shape
and do not contain `found`. Older engines return only `jobId` for an unknown id; clients should
treat a missing `jobStatus` as the compatibility fallback.

</details>

------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -798,18 +806,22 @@ This API has been deprecated, please use /job-info/:jobId instead
}
```

`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` always be returned.
`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` are returned for a job the cluster knows; an unknown id returns the `found: false` response below.
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` will return when job is running.
`finishTime`, `errorMsg` will return when job is finished.

When we can't get the job info, the response contains only the requested `jobId`:
An unknown id is still an HTTP 200 response with `found: false`:

```json
{
"jobId" : "733584788375666689"
"jobId" : "<id>",
"found" : false
}
```

See the `/job-info/:jobId` section in [REST API V2](rest-api-v2.md) for details and the
older-server fallback.

</details>

------------------------------------------------------------------------------------------
Expand Down
9 changes: 9 additions & 0 deletions docs/zh/engines/zeta/python-sdk.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,15 @@ with SeaTunnelClient(base_url="http://localhost:8080") as client:
print(job_details)
```

未知 id 仍会返回 HTTP 200,并在响应中带上 `"found": false`。如果只需要判断作业是否存在,
可以使用 `job_exists`;它也兼容旧版本引擎(旧版本对未知 id 只返回 `jobId`):

```python
with SeaTunnelClient(base_url="http://localhost:8080") as client:
if not client.jobs.job_exists(jobId=12345):
print("该作业 ID 不存在,或已从已完成作业历史中过期")
```

### 集群监控

您可以使用 `cluster` 属性来获取集群信息。
Expand Down
2 changes: 1 addition & 1 deletion docs/zh/engines/zeta/rest-api-job-lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -376,7 +376,7 @@ curl http://<master>:8080/overview | \
|---|---|---|
| 任意端点返回 `HTTP 404` | REST API 未启用或端口不对 | 设置 `enable-http: true` 并检查端口 |
| `Connection refused` | Master 未启动或防火墙拦截 | 确认 Master 进程在运行;检查防火墙 |
| `job-info` 中找不到 `jobId` | 作业已完成或未启动 | 按预期终态改用 `/finished-jobs/:state` 查询 |
| `job-info` 返回 `found: false`(或无状态的 200) | id 不存在,或作业已从已完成作业历史中过期 | 按预期终态改用 `/finished-jobs/:state` 查询 |
| 提交返回 `400 Bad Request` | JSON 格式错误或缺少必填字段 | 验证 JSON;检查 `plugin_name` 拼写 |
| `Job already exists with same job.id` | 相同 `job.id` 重复提交但未先停止 | 先取消或停止现有作业,再重新提交 |
| `Unauthorized 401` | 已启用 Basic Auth 但未提供凭据 | 添加 `-u user:pass` |
Expand Down
18 changes: 12 additions & 6 deletions docs/zh/engines/zeta/rest-api-v1.md
Original file line number Diff line number Diff line change
Expand Up @@ -419,7 +419,7 @@ network:
}
```

`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` 字段总会返回.
对于集群中存在的 Job,会返回 `jobId`、`jobName`、`jobStatus`、`createTime`、`jobDag`、`metrics` 字段;未知 id 会返回下方的 `found: false` 响应。
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` 字段在Job在RUNNING状态时会返回
`finishTime`, `errorMsg` 字段在Job结束时会返回,结束状态为不为RUNNING,可能为FINISHED,可能为CANCEL
运行中的Job还会返回 `diagnostics` 字段(状态时间戳与各 Pipeline 的恢复次数),字段说明见 [REST API V2](rest-api-v2.md)。该字段只在本接口返回,`/running-jobs` 不返回。
Expand All @@ -445,14 +445,17 @@ network:
| TableSinkWrite* | 按表汇总的 Sink 写入尝试,键格式 `TableSinkWriteXXX#<表>` |
| TableSinkCommitted* | 按表汇总的 Sink 已提交指标,键格式 `TableSinkCommittedXXX#<表>` |

当我们查询不到这个Job时,返回结果只包含请求的 `jobId`:
未知 id 仍会返回 HTTP 200,并带有 `found: false`:

```json
{
"jobId" : "733584788375666689"
"jobId" : "<id>",
"found" : false
}
```

完整说明及旧版本引擎的兼容方式请参阅 [REST API V2](rest-api-v2.md) 的 `/job-info/:jobId` 章节。

</details>

------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -508,18 +511,21 @@ network:
}
```

`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` 字段总会返回.
对于集群中存在的 Job,会返回 `jobId`、`jobName`、`jobStatus`、`createTime`、`jobDag`、`metrics` 字段;未知 id 会返回下方的 `found: false` 响应。
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` 字段在Job在RUNNING状态时会返回
`finishTime`, `errorMsg` 字段在Job结束时会返回,结束状态为不为RUNNING,可能为FINISHED,可能为CANCEL

当我们查询不到这个Job时,返回结果只包含请求的 `jobId`:
未知 id 仍会返回 HTTP 200,并带有 `found: false`:

```json
{
"jobId" : "733584788375666689"
"jobId" : "<id>",
"found" : false
}
```

完整说明及旧版本引擎的兼容方式请参阅 [REST API V2](rest-api-v2.md) 的 `/job-info/:jobId` 章节。

</details>

------------------------------------------------------------------------------------------
Expand Down
22 changes: 16 additions & 6 deletions docs/zh/engines/zeta/rest-api-v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -665,7 +665,8 @@ seatunnel:
}
```

`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` 字段总会返回.
对于集群中存在的 Job,会返回 `jobId`、`jobName`、`jobStatus`、`createTime`、`jobDag`、`metrics` 字段。
未知 id 会返回下方描述的 `found: false` 响应。
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` 字段在Job在RUNNING状态时会返回
`finishTime`, `errorMsg` 字段在Job结束时会返回,结束状态为不为RUNNING,可能为FINISHED,可能为CANCEL
`diagnostics` 字段在Job运行中且能从Master节点读取到诊断信息时返回。它属于辅助信息,读取失败时该字段会被省略,不会导致请求失败。该字段只在本接口返回,`/running-jobs` 不返回:为列表中的每个Job收集诊断信息会额外增加一次到Master的请求。
Expand Down Expand Up @@ -704,14 +705,20 @@ seatunnel:
| pipelines[].stateTimestamps | Pipeline 进入各状态的时间戳(毫秒)。发生恢复后,新一轮的时间戳会覆盖上一轮 |
| totalPipelineRestoreCount | Job 下所有 Pipeline 的 `restoreCount` 之和 |

当我们查询不到这个Job时,返回结果只包含请求的 `jobId`:
当查询的 Job 不存在时,仍然返回 HTTP 200,但响应中只包含请求的 id 和显式的
`found: false` 标记:

```json
{
"jobId" : "733584788375666689"
"jobId" : "733584788375093248",
"found" : false
}
```

`found: false` 表示当前集群没有保留该 id 的记录:可能是 id 写错了,也可能是作业已从已完成作业历史中过期。
成功查询的响应保持原有结构,不包含 `found` 字段。旧版本引擎对未知 id 只返回 `jobId`;客户端可以将
缺少 `jobStatus` 作为兼容判断。

</details>

------------------------------------------------------------------------------------------
Expand Down Expand Up @@ -786,18 +793,21 @@ seatunnel:
}
```

`jobId`, `jobName`, `jobStatus`, `createTime`, `jobDag`, `metrics` 字段总会返回.
对于集群中存在的 Job,会返回 `jobId`、`jobName`、`jobStatus`、`createTime`、`jobDag`、`metrics` 字段;未知 id 会返回下方的 `found: false` 响应。
`envOptions`, `pluginJarsUrls`, `isStartWithSavePoint` 字段在Job在RUNNING状态时会返回
`finishTime`, `errorMsg` 字段在Job结束时会返回,结束状态为不为RUNNING,可能为FINISHED,可能为CANCEL

当我们查询不到这个Job时,返回结果只包含请求的 `jobId`:
未知 id 仍会返回 HTTP 200,并带有 `found: false`:

```json
{
"jobId" : "733584788375666689"
"jobId" : "<id>",
"found" : false
}
```

完整说明及旧版本引擎的兼容方式请参阅 [REST API V2](rest-api-v2.md) 的 `/job-info/:jobId` 章节。

</details>

------------------------------------------------------------------------------------------
Expand Down
39 changes: 37 additions & 2 deletions seatunnel-cli/seatunnel_cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -1365,13 +1365,38 @@ def _poll_job_status(self, api_base: str, job_id: str):
poll_interval = 2
max_polls = 150 # 5 minutes max
last_status = ""
missing_response_seen = False

for _ in range(max_polls):
time.sleep(poll_interval)
try:
info = rest.request_json(f"{api_base}/job-info/{job_id}", timeout=10)

status = info.get("jobStatus", "UNKNOWN")
if isinstance(info, dict) and info.get("found") is False:
self.console.print(
f" [error]Job {job_id} is no longer present in the cluster's running "
f"or finished-job maps.[/error] Check the master logs; it may have been "
f"removed by a master restart without persisted state, or expired from "
f"the finished-job history."
)
return

status = info.get("jobStatus") if isinstance(info, dict) else None
if not isinstance(status, str) or not status.strip():
# A status-less 200 can be a transient master failover on older
# servers, so tolerate one before declaring the job unknown.
if not missing_response_seen:
missing_response_seen = True
continue
self.console.print(
f" [error]The engine returned no job status for job {job_id} twice."
f"[/error] This looks like a response-shape or master-failover problem; "
f"retry the command or check the master logs."
)
return

missing_response_seen = False
status = status.strip()
if status != last_status:
last_status = status
self.console.print(f" Status: [bold]{status}[/bold]", style="info")
Expand Down Expand Up @@ -1660,7 +1685,17 @@ def run_diagnose(job_id: str, console: Console) -> int:
# matches, and the output reads "Nothing to report" -- i.e. "your job is
# fine" about a job that does not exist. A non-object body (a bare `null`
# or a list from a proxy) is the same situation and is caught here too,
# before anything calls .get() on it.
# before anything calls .get() on it. Newer servers mark the same case
# explicitly with `found: false`.
if isinstance(job_info, dict) and job_info.get("found") is False:
console.print(
f"[error]No job {escape(job_id)} on the cluster at "
f"{escape(_ENGINE_API_BASE)}.[/error] The engine marked it as unknown: "
f"either the id is wrong, or the job has been evicted from the "
f"finished-job history."
)
return 1

status = job_info.get("jobStatus") if isinstance(job_info, dict) else None
if not isinstance(status, str) or not status.strip():
console.print(
Expand Down
Loading