Many "Failed to copy" errors occur using "juicefs sync" to migrate data from OSS to JuiceFS

[C2H5OHlife]

I'm using Fluid with JuiceFS to manage my AI model files (mainly multi-GB tensor files) which are originally stored in an OSS bucket. When running a DataMigrate CR to migrate the data into JuiceFS, a large number of files fail during synchronization with following errors:

<ERROR>: Failed to copy data of <file name> in <time duration>: unexpected EOF [sync.go:516]
<ERROR>: Failed to copy object <file name>: unexpected EOF [sync.go:602]

I've already tried these things without noticeable improvement:

• reducing thread number
• using --bwlimit option
• increase the buffer size of juice mount

Any suggestions would be greatly appreciated.

[yudin-s]

unexpected EOF during juicefs sync points to the read/write stream being cut while copying an object, which fits large OSS-backed files better than a metadata-only or cache-size issue.

Because your files are multi-GB tensors, I would first make the retry/resume path explicit and avoid the FUSE mount if the destination is JuiceFS:

juicefs sync --enable-checkpoint --debug \
  oss://.../prefix/ \
  jfs://META-URL/prefix/

Using jfs:// is worth testing for migrations into JuiceFS because it bypasses the mounted FUSE path. Increasing the mount buffer only helps if your destination is a mounted directory; it will not fix a read-side OSS EOF.

For the next run, I would also:

1. Keep --threads low, for example --threads=2 or --threads=4, until the EOFs stop.
2. Add --max-failure=-1 if you want the run to continue and collect all failing object names.
3. After the first pass, rerun the same command with --enable-checkpoint; the checkpoint should retry the pending/failed objects instead of starting over.
4. If the same objects fail repeatedly, test one failed object outside JuiceFS with the OSS/S3 tool from the same node. Repeated failure on the same key points to source object/network/credential behavior, while random failures point more to network/proxy timeout or bandwidth throttling.
5. For integrity, add --check-new once the copy is stable. It is slower, but useful for large model files.

If the EOF still appears with low threads and jfs://, the most useful follow-up is the exact juicefs version, the sync command with secrets removed, whether the same object names fail again, and a short --debug excerpt before the first EOF.

[C2H5OHlife]

Thanks for your reply and sorry for the late response. I've been testing juicefs sync with various combinations of options according to your suggestions, but the EOF errors still persist even using --threads=2, --max-failure=-1 and jfs:// as the destination(DST). Unfortunately --enable-checkpoint is not suitable for my use case since the juicefs sync runs in job pods which don't share a common checkpoint file. The failure point seems random, and every object can eventually be synced after enough retries(some objects require a large number of retries though). My JuiceFS version is v1.3.0.
My JuiceFS backend storage and the source OSS bucket are actually hosted on the same cloud storage provider, but what I find strange is the juicefs warmup is always fast and stable under the same environment.
Does this behavior seem reasonable, or does it suggest that the EOF errors may be related to something specific in juicefs sync rather than the underlying storage/network service?