Repository navigation
perf(maafw): 运行环境按项目指纹缓存,并在自动更新后确认一次 - #610
Merged
qiyinxi merged 7 commits intoSep 8, 2026
Merged
Conversation
编辑页每打开一次就调一次 /maafw/agent-env/prepare,前端的去重只活在组件本地 ref 里、页面一卸载就没了,后端也没有任何短路。于是每次进页面都要取项目锁、起 解释器核一次 ABI、再逐个 agent 跑 pip 健康检查——项目没动过时这几秒全是空转。 底层准备本身是幂等的(隔离 venv 靠自己的 manifest 三段哈希决定重不重建),所以 这不是在重下 MaaFramework,纯粹是重复确认。 现在把成功准备过的结果连同项目输入指纹落盘到 config/maafw_env_cache。指纹用 runner 包既有的 project_environment_fingerprint(),它哈希的正是 interface / requirements / uv.lock 这些「脚本更新了没」的输入,项目一更新缓存自然失效。命中 还要求结果里记的解释器与各 venv 路径都还在盘上——只做 stat,不验内容。 不验内容就意味着「文件还在、venv 内部已经坏了」会被放行,所以留了两条兜底:运行 前 check() 里的 describe_unusable_runtime() 仍会起解释器核一次 ABI;编辑页的重试 按钮带 force 直接绕过这层缓存。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
更新完只刷了 interface 缓存就往下跑,隔离 venv 的重建靠 runner 内部按需触发—— 那已经是 worker 子进程里、模拟器和游戏都起来之后的事,用户看到的只是任务卡住, 进度也只落在运行日志里。 现在 BeforeRun / AfterRun 两个时机的更新之后各接一次环境确认。更新失败也确认: 项目还是原样,环境该备还是得备。指纹没变时它只是一次哈希加几个 stat,所以「已是 最新」这条常见路径不额外花钱;AfterRun 顺手把下一轮要用的环境备好。 准备失败只记日志加通知、不抛出:runner 本来就会按需准备,这里失败最多把成本推回 运行时,不该让本来能跑的任务跑不了。受管项目(带 Store sidecar)跳过,它的环境由 Store/Gateway 那条链负责。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
复核出来的两处: 一、`_ensure_project_environment` 起的准备线程没有取消令牌。任务停止走的是 `task.cancel()`,它只让 `await asyncio.to_thread(...)` 抛出,工作线程里的 uv 子进程照跑不误——首次冷启动可能持续数分钟,界面上却已经显示任务结束;立刻重启 还会在运行池的锁上静默等到那个孤儿线程结束。`prepare_project_environment` 补上 `cancel_event` 透传给 `prepare_environment`(`prepare_agent_envs` 不可中断,签名 里注明),manager 侧照 `runner_task` 那条准备路径的做法:置位令牌加有界等待, 等不到就放手。 二、指纹只覆盖项目文件,不覆盖宿主引导解释器。运行池里 runtime 的身份含 `python_identity`,源头是本进程的 `sys.executable`;AUTO-MAS 换了随包 Python 之后 旧 runtime 还在盘上、项目也没动过,光比项目指纹会命中,而运行时会解析到另一份 runtime 并在任务里现建——正是这层缓存要避免的事。缓存改记宿主身份并纳入命中判据, 只读 `sys.executable` 与 `sys.version`,零成本;缓存格式版本升到 2,旧缓存自然失效。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
真机跑一次 BeforeRun 才发现的:`_append_update_log` 末尾写 `script_info.log`, 而那个 setter 会 `schedule_on_change()` → `asyncio.create_task`。在 `asyncio.to_thread` 的工作线程里调它直接 `RuntimeError: no running event loop`, 于是整段环境确认失败,只在日志里留下一句「运行环境确认失败,任务继续:no running event loop」——容错把异常吞了,功能一次也没真跑成。 项目更新那条路径同样把 `self._append_update_log` 交给核心包,而 `apply_package_transaction` 就跑在 `asyncio.to_thread` 里,是同一个既有隐患 (只在真的下载并应用更新时才会走到,所以一直没被撞见),一并改掉。两处现在都 走 `_threadsafe_update_log()`,做法与 `runner_task` 的 `send_runner_log` 一致。 实测:BeforeRun 下「更新 → 环境确认 → 用户任务」三步顺序正确,不再报错; 再跑一次落到「项目文件没有变化,运行环境沿用上次的准备结果」。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
浏览器里实测出来的:命中分支也推了一条 stage=ready 的 WS 进度,它和 HTTP 响应 抢着写编辑页那一行提示,谁后到谁说了算——推了反而把响应里的「运行环境已就绪, MaaFramework 5.13.0b6」盖成一句干巴巴的「已就绪」。命中本来就没有进度可言, 不推即可。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
审查者指南本 PR 为 MaaFW 运行环境准备引入带环境完整性校验的项目指纹缓存,并将自动更新后的准备工作移到 BeforeRun/AfterRun 流程中;同时支持取消和失败降级,修复线程日志回调的事件循环异常,并更新 API、前端调用及测试。 带缓存的 MaaFW 环境准备时序图sequenceDiagram
participant EditPage
participant API as AgentEnvPrepareAPI
participant Cache as EnvCache
participant Runner as MaaFWRunnerService
participant Disk as ProjectAndRuntime
EditPage->>API: prepare_maafw_agent_env(force)
API->>API: project_environment_fingerprint()
API->>Cache: load_prepared_environment(project, fingerprint)
Cache->>Disk: stat cached runtime and venv paths
alt cache hit and paths valid
Cache-->>API: cached result
API-->>EditPage: MaaFWAgentEnvPrepareData(cached=true)
else force or cache miss
API->>Runner: prepare_project_environment(...)
Runner->>Disk: prepare runtime and agent environments
Runner-->>API: prepared result
API->>Cache: store_prepared_environment(project, fingerprint, result)
API-->>EditPage: MaaFWAgentEnvPrepareData(cached=false)
end
自动更新前后环境准备时序图sequenceDiagram
participant Task as MaaFWTask
participant Update as ProjectUpdate
participant Prepare as EnvironmentPreparation
participant Runner as MaaFWRunnerService
participant Project as ProjectFiles
alt AutoUpdateMode=BeforeRun
Task->>Update: _run_project_update("BeforeRun")
Update-->>Task: update completed or unchanged
Task->>Prepare: _ensure_project_environment("BeforeRun")
else AutoUpdateMode=AfterRun
Task->>Update: _run_project_update("AfterRun")
Update-->>Task: update completed or unchanged
Task->>Prepare: _ensure_project_environment("AfterRun")
end
Prepare->>Project: project_environment_fingerprint()
alt cached environment available
Prepare-->>Task: reuse cached result
else environment requires preparation
Prepare->>Runner: prepare_project_environment(cancel_event)
Runner->>Project: rebuild or validate runtime and agent venvs
Runner-->>Prepare: preparation result
Prepare-->>Task: preparation completed
end
Note over Prepare,Task: Preparation failure is logged and does not block the task
MaaFW 环境缓存校验流程图flowchart TD
A[Project environment preparation requested] --> B["project_environment_fingerprint()"]
B --> C{force enabled?}
C -->|Yes| F["prepare_project_environment()"]
C -->|No| D["load_prepared_environment()"]
D --> E{Fingerprint, host Python, and paths valid?}
E -->|Yes| G[Return cached result]
E -->|No| F
F --> H["store_prepared_environment()"]
H --> I[Return prepared result]
文件级变更
提示与命令与 Sourcery 交互
自定义使用体验访问你的控制面板即可:
获取帮助Original review guide in EnglishReviewer's Guide本 PR 为 MaaFW 运行环境准备引入带环境完整性校验的项目指纹缓存,并将自动更新后的准备工作移到 BeforeRun/AfterRun 流程中;同时支持取消和失败降级,修复线程日志回调的事件循环异常,并更新 API、前端调用及测试。 Sequence diagram for cached MaaFW environment preparationsequenceDiagram
participant EditPage
participant API as AgentEnvPrepareAPI
participant Cache as EnvCache
participant Runner as MaaFWRunnerService
participant Disk as ProjectAndRuntime
EditPage->>API: prepare_maafw_agent_env(force)
API->>API: project_environment_fingerprint()
API->>Cache: load_prepared_environment(project, fingerprint)
Cache->>Disk: stat cached runtime and venv paths
alt cache hit and paths valid
Cache-->>API: cached result
API-->>EditPage: MaaFWAgentEnvPrepareData(cached=true)
else force or cache miss
API->>Runner: prepare_project_environment(...)
Runner->>Disk: prepare runtime and agent environments
Runner-->>API: prepared result
API->>Cache: store_prepared_environment(project, fingerprint, result)
API-->>EditPage: MaaFWAgentEnvPrepareData(cached=false)
end
Sequence diagram for environment preparation around automatic updatessequenceDiagram
participant Task as MaaFWTask
participant Update as ProjectUpdate
participant Prepare as EnvironmentPreparation
participant Runner as MaaFWRunnerService
participant Project as ProjectFiles
alt AutoUpdateMode=BeforeRun
Task->>Update: _run_project_update("BeforeRun")
Update-->>Task: update completed or unchanged
Task->>Prepare: _ensure_project_environment("BeforeRun")
else AutoUpdateMode=AfterRun
Task->>Update: _run_project_update("AfterRun")
Update-->>Task: update completed or unchanged
Task->>Prepare: _ensure_project_environment("AfterRun")
end
Prepare->>Project: project_environment_fingerprint()
alt cached environment available
Prepare-->>Task: reuse cached result
else environment requires preparation
Prepare->>Runner: prepare_project_environment(cancel_event)
Runner->>Project: rebuild or validate runtime and agent venvs
Runner-->>Prepare: preparation result
Prepare-->>Task: preparation completed
end
Note over Prepare,Task: Preparation failure is logged and does not block the task
Flow diagram for MaaFW environment cache validationflowchart TD
A[Project environment preparation requested] --> B["project_environment_fingerprint()"]
B --> C{force enabled?}
C -->|Yes| F["prepare_project_environment()"]
C -->|No| D["load_prepared_environment()"]
D --> E{Fingerprint, host Python, and paths valid?}
E -->|Yes| G[Return cached result]
E -->|No| F
F --> H["store_prepared_environment()"]
H --> I[Return prepared result]
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
按 tests/AGENTS.md,功能边界测试不提交,本地验证结果写进 PR 正文。这份 11 个 用例基本都是「指纹变了 / 路径没了 / 缓存损坏 / 格式版本不认识 / 宿主解释器换了」 这类边界,留在本地跑。文件仍在盘上,本地忽略写在 git-common-dir 的 info/exclude。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
摘要
项目配置页每打开一次就完整走一遍运行环境确认(取项目锁、起解释器核一次 ABI、逐个 agent 跑 pip 健康检查)。底层准备本就是幂等的,这几秒全花在「项目没变」上。现在按项目输入指纹缓存准备结果,指纹用 runner 包既有的
project_environment_fingerprint()(interface / requirements / uv.lock 等),项目一更新自然失效;命中还要求结果里记的解释器与 venv 路径都还在、宿主引导解释器身份未变。手动重试带force绕过这层缓存。自动更新更新完只刷了 interface 缓存就往下跑,隔离 venv 的重建靠 runner 内部按需触发——那已经是 worker 子进程里、模拟器和游戏都起来之后,用户看到的只是任务卡住。现在
BeforeRun/AfterRun两个时机各接一次环境确认;指纹没变时它只是一次哈希加几个 stat。准备失败只记日志不阻断运行;任务被取消时把令牌传进安装线程并有界等待,做法与runner_task那条准备路径一致。顺带修一个既有问题:
_append_update_log末尾写script_info.log,那个 setter 会asyncio.create_task,在asyncio.to_thread的工作线程里调直接RuntimeError: no running event loop。项目更新把它交给了核心包,而apply_package_transaction就跑在线程里(只在真的下载并应用更新时走到)。两处都改走loop.call_soon_threadsafe。本地验证
真机跑 MaaEnd v2.28.0-beta.1 项目(Windows,本分支后端监听 36174):
force=true浏览器里进出配置页实测,
agent-env/prepare往返 5.9ms / 18.5ms。运行流程(AutoProxy,
AutoUpdateMode=BeforeRun)日志顺序:第二次运行落到「项目文件没有变化,运行环境沿用上次的准备结果」。
检查项:
python -m pytest tests -q:489 passed, 3 skipped, 97 subtests passed;--collect-only -q退出码 0(492 collected)yarn typecheck/yarn lint:通过yarn test:唯一失败是electron/services/backendService.test.ts的 managed 用例超时,在 dev 基线上同样失败,与本 PR 无关frontend/src/api由openapi生成器重新生成,未手改另外本地为
env_cache写了 11 个用例覆盖上面这些失效判据(指纹变化、路径消失、缓存损坏、格式版本不认识、宿主解释器更换、per-project 隔离、discard 幂等),按tests/AGENTS.md属功能边界测试,不随 PR 提交。未验证:更新路径那处
call_soon_threadsafe修复要真的下载并应用一个更新包才会走到,本地没造出这个场景,是按代码路径判断的。🤖 Generated with Claude Code