[br] UnmarshalDir may panic after a cloud storage WalkDir error

[jebter]

Bug Report

1. Minimal reproduce step (Required)

1. Run BR v8.5.8 with a cloud object-storage backend and start a restore operation that reads migration or sub-compaction metadata through storage.UnmarshalDir.
2. During the metadata listing, let the object-storage WalkDir operation return a transient error after at least one metadata-read worker has already been scheduled. Examples include an object-storage 5xx response, a network timeout, a failed pagination request, or credentials being revoked during the listing.
3. Continue consuming the metadata iterator and observe the BR process.

The v8.5.8 runtime path can be reproduced with a storage test double that schedules one worker, returns a WalkDir error, and delays the worker's ReadFile/unmarshal operation until after WalkDir returns.

2. What did you expect to see? (Required)

BR should wait for all scheduled metadata workers, return the original object-storage error, and exit the restore operation cleanly. It must not panic while reporting an external-storage failure.

3. What did you see instead (Required)

The UnmarshalDir reader closes its result channel when WalkDir returns an error, without first waiting for the workers that were already submitted. A worker can subsequently send a parsed metadata value to the closed channel, producing:

panic: send on closed channel

This was directly reproduced in the v8.5.8 runtime path. The trigger is not limited to a synthetic invalid metadata format: a transient failure while listing cloud objects can provide the same interleaving. Depending on the call path, this can terminate BR during restore/PiTR metadata loading and leave the operation incomplete.

Relevant code: br/pkg/storage/helper.go, UnmarshalDir.

4. What is your TiDB version? (Required)

TiDB v8.5.8, commit 8b857efa20363d50a8fa2ea7dd9809a85a61b115.

Related change: PR #64850, which introduced the current BR compact-log-restore storage helper implementation.

[wjhuang2016]

Version triage

Affected release lines confirmed by source comparison: 26.3.

UnmarshalDir skips joining workers when WalkDir fails, and its consumer can interpret a closed result channel as clean EOF while an error is queued.

For the releases below, the complete enclosing production functions changed by #71139 are byte-identical to the pre-fix functions at e73d535e34046990be05f689bdde1380222c0069, not merely a matching filename or isolated line. This confirms the same faulty implementation in these release snapshots.

Release	Inspected source
26.3	func UnmarshalDir[T any](ctx context.Context, walkOpt *storeapi.WalkOption, s storeapi.Storage, unmarshal func(target *T, name string, content []byte) error) iter.TryNextor[*T]

The remaining may-affects labels (7.5, 8.1, 8.5, 25.10) are intentionally retained: those implementations differ and need separate diagnosis. A source mismatch alone is not evidence of being unaffected.

This is source-based diagnosis; historical-release runtime tests were not rerun. Regression/fix evidence is in #71139. Existing severity and already-confirmed affects labels are preserved. This does not claim merge or backport completion. Where present, 25.10/26.3 denote release-nextgen-20251011/release-nextgen-202603.