fix(powermetrics): time out the powermetrics subprocess - #1397
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1397 +/- ##
=======================================
Coverage 91.43% 91.44%
=======================================
Files 49 49
Lines 5057 5073 +16
=======================================
+ Hits 4624 4639 +15
- Misses 433 434 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Some change made:
I deliberately left the returncode != 0 path returning True (warn, then read the file) — that's today's behaviour, and flipping it would silently turn real readings into zeros for anyone whose powermetrics exits non-zero but still writes data. Say the word if you'd rather that path also skip. One thing this does not fix, and should be treat as a separate issue: when |
subprocess.call had no timeout, so a hung powermetrics blocked the measurement thread forever. Allow twice the expected sampling duration plus a startup margin.
1. The timeout now actually bounds the hang. Replaced subprocess.call(..., timeout=) with an explicit Popen + wait(timeout=), deliberately not under a context manager, since Popen.__exit__ calls wait() with no timeout and would re-hang when the kill is refused. Added stdin=subprocess.DEVNULL so sudo gets EOF and exits non-zero instead of blocking on a password prompt — that removes the main hang at the source rather than relying on being able to kill a root-owned child.
2. New _kill_process() helper that swallows OSError/SubprocessError from kill()/wait(), so a refused signal can't turn the handled TimeoutExpired into an escaping PermissionError, and closes the pipes the context manager used to close.
3. Applied the same fix to _has_powermetrics_sudo(). It had the identical bug — a return False from inside with Popen(...) after a possibly-failing process.kill(), i.e. an unbounded wait() in __exit__ during startup probing. It now uses a plain Popen, stdin=DEVNULL, and _kill_process(). The sudo-prompt detection still works: with DEVNULL, sudo writes "a terminal is required to read the password" to stderr, which the existing regex matches — just immediately instead of after the 3 s deadline.
4. The skip is now a real skip. _log_values() returns bool, and get_details() returns {} when it's False instead of re-reading a log file that still holds the previous measure. Also {timeout:g} so the warning reads "7 seconds", not "7.0 seconds".
I deliberately left the returncode != 0 path returning True (warn, then read the file) — that's today's behaviour, and flipping it would silently turn real readings into zeros for anyone whose powermetrics exits non-zero but still writes data. Say the word if you'd rather that path also skip.
One thing this does not fix, and I'd treat as a separate issue: when get_details() returns {}, AppleSiliconChip._get_power() records 0 W for that interval rather than omitting the sample. There's no "no measurement" concept at that layer, so a genuine skip needs a change in hardware.py.
4ce030c to
9720594
Compare
Split out of #1333, which also rejects powermetrics on non-Apple-Silicon Macs. This is just the subprocess timeout, separated so it can land independently.
ApplePowermetrics._log_values()shells out withsubprocess.call(cmd, universal_newlines=True)and no timeout. The command issudo powermetrics ..., so if sudo has no cached credential and no tty to prompt on, or if powermetrics itself wedges, the call blocks forever. It is invoked fromget_details()on every measurement, which means the tracker's measurement path hangs with no output and no way out short of killing the process.This passes a timeout and, on
subprocess.TimeoutExpired, logs a warning and returnsNoneso that measure is skipped rather than blocking. The existing non-zero-returncode path is unchanged.The timeout value is a heuristic, not a measured bound:
_n_pointssamples at_intervalmilliseconds each is the nominal runtime of the command, son_points * interval / 1000is that runtime in seconds. It is doubled to absorb sampler overhead and scheduling jitter, and 5 seconds is added as a floor so that short configurations still get a usable margin. With the defaults (n_points=10,interval=100) that is 1 second of expected work and a 7 second limit.Nothing measured picks the 2x and the +5; they are chosen to be comfortably loose so a healthy run never trips them, and the timeout only exists to bound a hang. If a reviewer has real numbers for how long powermetrics overruns its nominal duration under load, that is the input that should replace this formula.
tests/test_powermetrics.pygains a case asserting_log_values()returnsNoneand warns once whensubprocess.callraisesTimeoutExpired.uv run pytest tests/test_powermetrics.pypasses (17 tests).