feat(fs): support skipping existing files in copy tasks#9557
Open
okatu-loli wants to merge 1 commit into
Open
Conversation
Add skip_existing option to POST /fs/copy. When enabled, destination files with the same name and size are skipped instead of failing the request, so an interrupted cross-storage copy can be resumed without re-transferring completed files. Files with mismatched sizes (e.g. truncated by a previous interruption) are re-copied.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #9555
问题
跨网盘复制(如 pikpak → 115)任务因关机/网络问题中断后,重新发起复制时
/api/fs/copy只有overwrite一个选项:不勾选则因目标已存在同名直接报错,勾选则所有文件全量重传,已完成的部分被浪费。方案
为
POST /api/fs/copy新增skip_existing参数:skipped: destination file already exists);不依赖 MD5/哈希校验——各网盘驱动哈希算法不一致或缺失,跨盘场景大多不可用;同名同大小判定可同时覆盖断点续传与残缺文件重传两个诉求。
实现
MoveCopyReq增加skip_existing字段,FsCopy在该参数开启时放行已存在的目标;conf.NoTaskKey模式)传入fs.Copy,签名不变,既有调用点零改动;CopyTask.SkipExisting带 json tag 持久化,进程重启恢复任务后标志不丢,目录递归生成子任务时逐层传递;copyFileBetween2Storages在请求源文件下载链接前探测目标,命中同名同大小即跳过;探测失败(网络等原因)退化为照常复制,最坏多传一次,不会使任务失败;测试
本地两个 Local 存储间实测:
skip_existing、overwrite=false复制已存在目录 →403 file [dir] exists(回归,行为不变);skip_existing=true复制目录(目标预置同大小文件、半截文件、缺失文件)→ 同大小文件跳过且内容未被覆盖,半截文件重传为完整内容,缺失文件正常复制;配套前端 PR:AlistGo/alist-web(复制对话框新增「跳过已存在文件」选项)。