Skip to content

fix(python): a top-level call of a harmless same-module function keeps the package namespace - #2713

Merged
DeusData merged 6 commits into
mainfrom
fix/py-namespace-effect-allowlist
Oct 20, 2026
Merged

DeusData merged 6 commits into
mainfrom
fix/py-namespace-effect-allowlist

Conversation

@DeusData

Copy link
Copy Markdown
Owner

Stacked on #2712; this PR's base is its branch, and it is retargeted to main once #2712 merges.

What

#2704 publishes a Python package's re-exports from the module's namespace facts. An unknown statement in those facts withdraws every re-export recorded before it. A top-level call is such a statement, because it can rebind any name of the module. sqlalchemy ends three of its packages with one:

# lib/sqlalchemy/sql/__init__.py (likewise sqlalchemy/__init__.py, sqlalchemy/orm/__init__.py)
from .expression import and_ as and_
...
def __go(lcls: Any) -> None:
    from .. import util as _sa_util
    from . import base
    ...
__go(locals())

sql.cast, sql.select and sql.and_ therefore never reached the export table. They resolved by name guesses to the methods ColumnElement.cast, FunctionElement.select and BooleanClauseList.and_.

This change (extract_imports.c) keeps the namespace through a top-level call only when the callee provably cannot rebind the caller's names:

  • the callee is the latest binding of its name: a proven, undecorated function of the same module, with no star import or unknown effect since its definition;
  • every argument is a safe expression, or a bare locals() / globals() the module does not rebind;
  • the body, nested scopes included, has no global / nonlocal, and never names:
    • a parameter;
    • a function or class this module defined earlier (it could rebind through global);
    • the module's own name;
    • a name that reaches a namespace: globals vars locals exec eval setattr delattr __import__ sys importlib __name__ __dict__ __globals__ __builtins__ builtins __self__ inspect f_globals f_locals f_back gc.
  • The body may import. Each submodule of the package it imports becomes a package attribute, so in a package __init__ those names are shadowed at the call.

Every other call stays a whole-namespace effect. Other languages are unchanged.

Evidence

Clean builds; base = #2712's head (docstrings and __future__ facts), head = this branch; par, repeat 2, noise 0.

repo moved
sqlalchemy CALLS +55 / -55, all same-line retargets; 97 same-target strategy upgrades; nothing else
sentry, ansible byte-identical
  • The 55 retargets are exactly the sql.* calls the barrier held:
    • sql.and_ ×23 → _elements_constructors.and_;
    • sql.select ×22 → _selectable_constructors.select;
    • sql.cast ×7 → _elements_constructors.cast;
    • sql.any_ ×2, sql.or_ ×1.
  • Judged with snippets (seed 20261017, 12 per stratum): 12/12 added edges correct; 12/12 removed edges were wrong guesses (methods of other classes, or another module's function).
  • The 97 upgrades keep their target and move from import_map_suffix to lsp_module_attr: bindparam, text, null, case, literal_column, column, tuple_, orm.base.class_mapper, and others.
  • Namespace facts (replayed with fix(python): resolve names re-exported through package namespaces #2704's fold): all three __go packages now publish. In sqlalchemy, re-exported names that other files import from the module fall from 572 to 258 withdrawn, and importer/name pairs from 4,888 to 464.

Controls on the real files. Across the three repos exactly 7 top-level calls change their facts, and each was read on its source:

  • sqlalchemy's three __go(locals()) calls. Each is shadowed exactly by the submodules its body imports (for sql: base, coercions, elements, lambdas, selectable, schema, traversals, type_api, annotation).
  • sentry constants._load_platform_data(): mutates a module dict in place and calls an imported function; it rebinds no name.
  • sentry identity._register_providers(): nine from .X.provider import ... lines, which give nine correct subpackage shadows; it calls an imported register.
  • Two ansible test-module main() calls, which have no consumers.

None of these calls rebinds a name.

Full payoff with F13. Most from sqlalchemy import select imports are still bound by the import resolver's project-wide name guess (WriteOnlyCollection.select) before the export table is consulted. The follow-up that binds a Python from-import inside the module it names turns the published names into retargets: of sqlalchemy's 1,091 IMPORTS edges to a method, 1,046 are predicted to retarget with this PR in place, against 429 without it.

Tests

  • extraction: python_namespace_facts_init_call_allowlist:
    • the sqlalchemy __go shape: the namespace survives, and the imported submodules are shadowed at the call;
    • a plain module (no package attributes, so no shadows);
    • twelve controls that stay whole-namespace effects: the body uses its parameter; global (also nested); globals(); sys.modules[__name__]; the own package name; a same-module helper; an imported callee; a rebound callee; a call before the definition; an argument with an effect; a rebound locals.
  • pipeline: pipeline_python_package_reexports_resolve_by_lsp adds the __go package (a re-export resolves; a submodule the body imports is excluded) and a __go that writes its parameter (excluded), across both drivers × full and incremental.
  • RED before the change; GREEN with it.
  • The 31 affected suites pass on this branch (fix(python): docstrings and __future__ imports are no namespace barriers #2712 merged in): 4,120 tests, 0 failures (clean ASan/UBSan test-runner).
  • lint-memory-core reports no growth; cppcheck 2.20 and clang-format are clean.

sqlalchemy's sql/__init__.py ends with __go(locals()). The callee is a same-module function that never touches its parameter or the module's globals; only the submodules its body imports become package attributes. The extraction test pins the exact shape (the namespace survives, the imported submodules are shadowed at the call) and its controls: the body uses the parameter, global (also nested), globals(), sys.modules[__name__], the own package name, a same-module helper; an imported, rebound or not yet defined callee; an argument with an effect; a module-level locals. The pipeline fixture adds the same shape end to end, with a re-export the body's submodule import replaces and a __go that writes its parameter.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…s the namespace

A top-level call withdrew the whole namespace (UNKNOWN, reason EFFECT): sqlalchemy's sql/__init__.py ends with __go(locals()) and lost all 108 re-exports. The call is now allowed when the callee is the latest binding of its name, a proven same-module function with no star import or unknown effect since; every argument is a safe expression or a bare locals()/globals() the module does not rebind; and the body (nested scopes included) has no global/nonlocal and never names a parameter, a function or class the module defined earlier, the module's own name, or a name that reaches a namespace (globals, vars, locals, exec, eval, setattr, delattr, __import__, sys, importlib, __name__, __dict__, __globals__, __builtins__, builtins, __self__, inspect, f_globals, f_locals, f_back, gc). Each submodule of the package the body imports becomes a package attribute and is shadowed at the call; in a plain module the imports install nothing. Every other call stays a whole-namespace effect. The function table and verdicts live in the extraction scratch; each callee is checked once.

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
WHY: the check read an import's module path with strcspn, which runs to a NUL, but a tree-sitter source slice need not be NUL-terminated: the first segment is now found with memchr inside the slice. The leading dots move into py_ns_leading_dots; written inline, cppcheck 2.20 (the CI version) took the loop to always end at the slice length and reported three knownConditionTrueFalse findings on the branches that follow it.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…ace-effect-allowlist

Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData
DeusData changed the base branch from fix/py-namespace-string-future-facts to main October 19, 2026 22:26
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
@DeusData DeusData added bug Something isn't working parsing/quality Graph extraction bugs, false positives, missing edges labels Oct 19, 2026
@DeusData
DeusData merged commit 4c80d89 into main Oct 20, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working parsing/quality Graph extraction bugs, false positives, missing edges

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant