Preserve GLM-5.2 Megatron accuracy-compatible specs - #3
Draft
zrr1999 wants to merge 22 commits into
Draft
Conversation
Resolve swift/megatron/init.py: keep both _patch_mcore_bridge additions. Upstream main added the import-time use_accuracy_compatible TE-off patch; this branch added the norm_accuracy_compatible DSA q/kv layernorm spec patch. They are independent and both are applied, TE-off first.
…ed by this branch
_replace_spec_dsa otherwise still asks TESpecProvider, leaving TELinear on indexer/MLA while Paddle HAVE_TE is False.
Stack-top dump-off YAML uses those keys. Without the dataclass fields parse_args left remaining_argv and failed closed. Alias MTP depth onto mtp_num_layers and apply the Megatron deterministic contract at init.
TE-off local TransformerLayer has a standalone pre_mlp_layernorm, not fused linear_fc1.layer_norm_weight. Weight load asserted None on that fused key. Route the HF post-attention LN onto the local module when the fused parameter is absent.
TP2+SP pads ceil(57/2)*2=58 without the extra *2, while PaddleFleet and the E-811 IEEE 1-100 carrier are 60. Restore the SP interleave factor so get_padding_to is 4 on the frozen profile. Signed-off-by: Zhan Rongrui <me@zrr.dev>
CI flake8/isort/yapf on PFCCLab#3 failed on the new test imports and nearby wrap. Numerics unchanged. Signed-off-by: Zhan Rongrui <me@zrr.dev>
Merge PFCCLab/ms-swift main so lint --all-files includes the new alignment workflow and build.sh. Wrap the two E501 lines in the observability trainer helpers and let isort/yapf/single-quote hooks rewrite only files already on this branch.
CI pre-commit --all-files still ran double-quote-string-fixer on utils.py after the main merge. Local --all-files is now green.
The previous quote-fixer pass rewrote rf'({"|".join(...)})' to
' | ', which would change the regex. Restore the original
alternation and only single-quote the dump/load helper strings
that CI's double-quote-string-fixer still rewrote.
CI lint uses Python 3.10, whose tokenize still sees f"..." as STRING tokens, so double-quote-string-fixer rewrites them. Local 3.12 tokenize splits f-strings and the hook no-ops. Convert the four logger f-strings and one comment on this branch.
Self-hosted unittest died at sudo chown with "no tty present and no askpass". Keep chown when sudo -n works; otherwise skip so checkout can run.
Keep pull_request on the historical CodeSync/develop tarball and develop/latest wheels. workflow_dispatch can opt into fail-closed SHA checkout plus artifact digest checks; pairing remains unproven unless this job built the wheels from the pin. MinimaxV2.5_EP2 and GLM45Air_EP2 are unchanged.
paddle-4 runner 2.337.0 forces Node 24 for actions/checkout@v3; node24 needs GLIBC 2.27/2.28 which this host lacks (job 101275701451). Keep checkout@v3 and opt into ACTIONS_ALLOW_USE_UNSECURE_NODE_VERSION.
Match Megatron PFCCLab#4: checkout COMMIT_ID on workflow_dispatch, export source trees from the pin selector, and consume that env in the alignment step so setup_venvs never sees the hardcoded 0.0.0 wheel.
Match Megatron PFCCLab#4: caller mode/pin stay authoritative; unproven develop fallback is refused; verified 0.0.0 names remain legal.
Selector clone failure must fail the docker exec (set -e) instead of continuing wget/build. Receipt checks live in a helper so the single-quoted -c script has no nested quotes. Missing pin.env after an error receipt is selector failure, not env-handoff.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scope
ms-swift accuracy-compatible TE-off: after forcing local decoder/MTP specs, also replace
_get_backend_spec_providerwithLocalSpecProviderso DSA indexer / MLA are not still built as TELinear/TENorm.Evidence
post_attn_norm29/30 → 5/30; PaddleHAVE_TEis False.venvs/torch/bin/python, venv cuDNN first):tests/megatron/test_model_config.py::test_dsa_backend_forced_to_local_spec_when_accuracy_compatible— 1 passed.Gate state
Not accepted. Source freeze blocked. This PR's evidence is probe-class and does not advance a gate.
Blocked by
contract_environment.source-freeze-head-vs-pinacceptance_oracle.torch-results-checkpoint-emptyNot verified
use_accuracy_compatibleis on).