SM8550 deep suspend/resume

[shuuri-labs]

Summary

This PR enables the deep suspend path (S2RAM) for SM8550, tested on the Retroid Pocket 6.

It is @jaewun's #2952 series, cherry-picked with authorship intact and rebased onto current next (7.1.2). Rebase adjustments, noted in the commit messages:

• 0204 (ICE iface clock votes) is dropped, it landed upstream verbatim in 7.1.2
• 0201 has one hunk rebased, ufshcd_intr() gained the active_uic_cmd thread-handoff condition upstream
• the ICE/inline-crypto set is dropped entirely per the discussion below (not load bearing for suspend), so this PR no longer touches the kernel config

So this is the same suspend content as #2952, in a form that merges cleanly on next today.

Two commits are mine:

• the IPCC mailbox fix, split back out with its original commit. This is the wake fix the RP6 needs: the ADSP charger firmware pushes an unsolicited BATTMGR_NOTIFICATION (opcode 0x7) about 0.5s after suspend entry, and with IRQF_NO_SUSPEND the mailbox irq is never masked, so the device wakes itself every few minutes. Lifted from ROCKNIX's SM8750 patch, which shares the battmgr/pmic_glink charger model.
• broaden the TSENS uplow-wake skip from ayn,thor to qcom,sm8550. The RP6's compatible list has no ayn,thor entry, so the Thor-only check silently does nothing there and the TSENS threshold IRQs keep waking the device. The SoC-level compatible covers every SM8550 board, Thor included. Critical thermal wake stays enabled.

s2idle stays disabled (mem_sleep_default=deep plus SuspendState pinned to mem): a failed deep attempt that falls through to s2idle has wedged SM8550 devices in testing, so a failed deep should no-op instead.

Compared to the previous revision of this PR: the old patch lineage is replaced by the #2952 versions (which add the mphy hibern8-park and RX LineCfg fixes), and the rsinput MCU suspend/resume and geni UART patches are dropped to match #2952. Given the controller irq-storm behaviour jaewun describes in the comments, those three are the likely re-add candidates once we have his notes and an RP6 repro; keeping them out for now so the fix can be evidence-based.

Testing

Built and tested on the Retroid Pocket 6: repeated RTC-woken deep suspend/resume cycles, including longer unattended runs. Wake sources checked via /sys/power/pm_wakeup_irq (with PM_SLEEP_DEBUG enabled in my local builds) and /sys/kernel/debug/wakeup_sources, real time-in-suspend confirmed via CLOCK_BOOTTIME vs CLOCK_MONOTONIC drift.

Note: that testing was against the previous revision of this branch. The kernel patches here are byte-identical to what I build daily against stock 7.1.2, but a full rebuild and retest of this exact set on the RP6 is still to come, including the buttons-during-suspend controller repro. I'll report back here.

Thor coverage is jaewun's testing in #2952.

Test results

Deep suspend resumes reliably on the tested RP6 setup with no self-wakes. Before the IPCC fix the device could not stay asleep past 5 to 10 minutes; with it, clean ~14 minute RTC-bounded suspends with matching boottime drift. UFS recovers cleanly after resume. One UFS host reset per deep resume is expected on this platform because system suspend uses LINK_OFF.

Additional context

This PR and #2952 are the same series. The difference is the rebase onto current next, the ICE drop, and the two RP6-facing commits. Happy to fold this into #2952 instead if that's preferred.

AI Usage

Did you use AI tools to help write this code? Yes.

AI tools were used during the investigation and packaging: correlating wakeup-source and suspend-stat logs to pin the wake to the opcode 0x7 IPCC push, comparing candidate patches across the SM8550 and SM8650 trees, and assembling this branch. The IPCC fix itself is lifted verbatim from ROCKNIX's SM8750 patch, and the final selection was validated by hardware testing on the RP6.

[dcu]

awesome! I can help testing if you provide steps to test
do you have a pre built kernel somewhere?

[loki666]

also can you guys align
what's the difference with @jaewun #2952 PR ?

[Cyfarw9dd]

Thanks for your work! Already testing on an Odin2 portal — the suspend patch works so far, going to run a few more tests (repeated suspend/resume, wake stability etc).

[dcu]

Works really well for me as well RP6.

just once after resuming the device did not recognize the input but after sleeping and resuming again it worked. I haven't been able to reproduce it ever since.

EDIT: I lost input recognition again, after suspending and resuming again it's restored. there must be some kind of race condition

[dcu]

@shuuri-labs can you rebase your branch on origin/next?

[jaewun]

I'm pretty sure we can drop the ICE patches to simplify things here, they were introduced early on in testing as a correctness patch but in retrospect they dont appear load bearing.

I also ran into the controller issues mentioned above and it seems like the controller continues sending interrupts, and storms a whole lot of interupts during wake and the irq eventually gets disabled as "nobody cared". Might be able to replicate by pressing buttons during suspend/wake? Will dig up my old notes on that when I get home next week.

[shuuri-labs]

@jaewun sweet, thanks for letting me know. I've dropped the ICE set from the branch (the crypto configs went with it, so this PR no longer touches the kernel config at all).

As for the controller - I originally carried your MCU suspend/resume + geni UART mask patches and that's mostly what I've tested, so it makes sense that I haven't really seen this issue yet. I removed them when rebasing off your clean set, so at some point I can run the buttons-during-suspend repro on the RP6 and report what pm_wakeup_irq and the irq counters show with the new set. Depending on the results, we can either re-add your older patches or something more targeted (from your your notes)

EDIT: actually, since @dcu reported facing the controller issue before my rebase, the now dropped patches probably didn't help. I'll do my investigation anyway, but probably we'll just wait for your notes for a fix :)

[dcu]

EDIT: actually, since @dcu reported facing the controller issue before my rebase, the now dropped patches probably didn't help. I'll do my investigation anyway, but probably we'll just wait for your notes for a fix :)

I have not seen those issues again though, not sure what changed or it is just pure luck
I've also tested pressing buttons during the suspend/resume phases and everything's ok, I have not seen those "nobody care" messages in dmesg
wifi speed is also good after waking up

[tiopex]

Tested on the Ayaneo Pocket EVO, installed on internal storage. It works in ES, but it still wedges fairly often when running Steam

I tested also on Ayaneo pocket DS installed on SD card. Sometimes it cannot enter into sleep with log
[ 170.891984] dwc3-qcom a600000.usb: port-1 HS-PHY not in L2 [ 170.912507] xhci-pci-renesas 0001:01:00.0: PM: pci_pm_suspend(): hcd_pci_suspend returns -16 [ 170.912580] xhci-pci-renesas 0001:01:00.0: PM: dpm_run_callback(): pci_pm_suspend returns -16 [ 170.912631] xhci-pci-renesas 0001:01:00.0: PM: failed to suspend async: error -16 [ 170.916634] PM: Some devices failed to suspend, or early wake event detected [ 171.248975] usb 1-1: reset full-speed USB device number 2 using xhci-pci-renesas [ 171.517244] usb 1-2: reset full-speed USB device number 3 using xhci-pci-renesas [ 171.968676] input: Microsoft X-Box 360 pad as /devices/platform/soc@0/1c08000.pcie/pci0001:00/0001:00:00.0/0001:01:00.0/usb1/1-2/1-2:1.0/input/input14 [ 171.968694] OOM killer enabled. [ 171.968723] Restarting tasks: Starting [ 171.972899] Restarting tasks: Done [ 171.973006] random: crng reseeded on system resumption [ 171.973022] PM: suspend exit

No wedges here, but sometimes I have to push power button few times to wakeup the device

Tomorrow I'll check on DS installed on internal storage and on AYN thor

[tiopex]

After installing to internal on Pocket DS also wedges when running steam, similar to pocket EVO

[shuuri-labs]

@tiopex cool, thanks for checking. waiting on the Thor test. If that goes OK, then perhaps this should be gated to Odin 2 family/RP6/Thor for now, as the patches have been validated on all of these devices.

[tiopex]

tested on AYN Thor on SD card. Sometimes after wakeup buttons doesnt work and such error is visible in dmesg
dmesg.txt

[shuuri-labs]

@tiopex
Two changes on the branch:

I've gated deep suspend is to validated devices: RP6, Odin 2 family, and Thor. On the Ayaneo Pocket EVO and DS your reports show separate problems (the Steam wedges, and the xhci -16 refusing to enter sleep on the DS), so those devices keep suspend disabled for now. They can be added back once those issues get their own investigation.

As for the Thor button dropot: the trace shows the gamepad MCU still streaming into its UART while the kernel is suspending that port. There is a window in the teardown where the interrupt fires but the driver no longer services it, so the line refires endlessly until the kernel's spurious-IRQ protection gives up and disables it permanently (irq 190: nobody cared in your log). Nothing re-enables that line on resume, which is why buttons stay dead until reboot. This is the exact failure @jaewun originally wrote two patches for, so I've restored them both: one quiesces the MCU before the UART suspends, the other masks the geni irq across the suspend transition so the storm can't trip the detector. If you get a chance to retest on Thor (ideally pressing buttons around suspend entry, which widens the race window), that would confirm it.

Thank you for testing + providing logs :)

[tiopex]

Checked the latest. Buttons works fine now, but suspend sometimes takes long time to enter into sleep. It is visible also in log
[ 427.361885] PM: suspend entry (deep)
[ 481.492344] Filesystems sync: 54.130 seconds

54s for sync filesystem before entering into sleep.

Another issue is that sometimes need to push PWR button few times to wakeup the device

[Cyfarw9dd]

@shuuri-labs After testing this PR on Odin2 Portal: the kernel-side changes are applied correctly.

I'm still hitting a Steam Big Picture freeze on suspend/resume, but preliminary analysis suggests this is unrelated to this PR — on my device system.suspendmode = off (fake-suspend), so the power button goes through rocknix-fake-suspend instead of actual kernel s2ram. The freeze appears to be a fake-suspend + gamescope interaction bug.

Note: can't resume from deep suspend by controller button, only can wake up by power button.

SM8550:~/home/Isabel/GenericAgent # dmesg | grep -E "suspend|deep"
[    0.000000] Kernel command line: boot=LABEL=ROCKNIX disk=LABEL=STORAGE quiet rootwait console=tty0 allow_mismatched_32bit_el0 fw_devlink.strict=1 pcie_ports=compat irqaffinity=0-2 cgroup.memory=nokmem,nosocket nosoftlockup usbcore.interrupt_interval_override=045e:028e:2 ufshcd_core.uic_cmd_timeout=3000 mem_sleep_default=deep
[ 1613.600721] PM: suspend entry (deep)
[ 1613.647859] printk: Suspending console(s) (use no_console_suspend to debug)
[ 1615.642024] PM: suspend exit
[ 2517.508357] PM: suspend entry (deep)
[ 2517.669266] printk: Suspending console(s) (use no_console_suspend to debug)
[ 2519.749219] PM: suspend exit
SM8550:~/home/Isabel/GenericAgent #

[dcu]

can this be merged for now? whatever bugs there are this is better than the alternative

[xiaodoudou]

Tested the whole series on an AYN Odin 2. Built from tag 20260801 with this branch merged in, nothing else on top.

13 deep suspend cycles: 3 x 30s, 3 x 120s, 1 x 900s, 1 x 60s, 2 x 120s for a WoWLAN comparison, 3 by power key. 12 clean, 1 abort I caused myself.

All 13 entered deep, no s2idle fallthrough, SuspendState=mem held. No UFS errors, no hibern8 or link-startup retries. No nobody cared and nothing disabled by the spurious-IRQ detector. The 900s alarm slept 903s, so nothing self-woke it. tsens wake irqs: 0.

Sync and the suspend path are quick, no sign of the slow sync reported earlier in the thread:

[  613.218823] PM: suspend entry (deep)
[  613.241378] Filesystems sync: 0.022 seconds

Entry to exit was 2.2s to 6.1s.

1004 and 1006: the MCU is re-initialised on every resume and the pad keeps working.

[  525.014178] rsinput serial1-0: MCU Version: V02.04.001.01 2024-04-08 11:19:43

A couple of Frame too short for checksum validation / Checksum mismatch lines around the re-init, then it settles.

WiFi reloads firmware on every resume (mhi mhi0: Requested to power ON, then fw_version 0x110cffff). Time to reassociate: 8, 9, 9, 10, 10, 10, 10 and 24 seconds, independent of how long it slept. On 20260701 with kernel 7.0.11 I had anything from 7s to two minutes of scan failures, and one cycle that stayed dead through a WiFi toggle and a reboot. None of that happened here. Nothing in this PR touches WLAN resume, and WoWLAN no longer prevents the reload the way it did on 7.0.11.

One failed suspend, self-inflicted: I pressed power again before the resume had finished.

[  619.589889] Wakeup pending. Abort CPU freeze
last_failed_errno=-16

One consequence on my side: with real suspend working, my external display PR (#3080) will need updating, since it currently leans on the fake-suspend path. It has to be reworked on top of this series and tested against it. I plan to do that. In the meantime I am tracking battery consumption over the next 10 hours and will report back.

[xiaodoudou]

Battery numbers from the same build, AYN Odin 2 left alone overnight on battery.

One suspend, 9h 47m in deep sleep, nothing woke it in between (suspend_stats/success went 15 to 16 over the window). Battery went from 95% to 89%, so 6 points off an 8000mAh pack over 9.78h, around 50mA or 0.6% per hour. That projects to roughly a week of standby from full. The gauge only reports whole percent, so the honest range is 41 to 57mA.

It really was down rather than idling awake:

uptime (boottime) = 38745s
last dmesg timestamp (monotonic, frozen in suspend) = 1229s

so 10h 25m of the 10h 46m since boot was spent suspended, with about 20 minutes awake in total including all of last night's testing.

Resume after ten hours was clean: wlan0 back up, gamepad still enumerated, and since boot nobody cared / Disabling IRQ count 0 and ufshcd error count 0. It woke on a single power button press.

[xiaodoudou]

Retested on my Odin 2 against current next (833140e) plus this PR, and the suspend side looks good: 18 cycles, success went up every time, fail stayed at 0, always PM: suspend entry (deep). No UFS errors, gamepad fine coming back.

I did run into something while stress testing though, and since it's only reachable now that real suspend works, it seems worth writing up.

WiFi sometimes doesn't come back, and then you're stuck

Every so often after a resume, the WCN7850 reloads its firmware like normal and then just refuses to scan, forever:

mhi mhi0: Requested to power ON
mhi mhi0: Wait for device to enter SBL or Mission mode
ath12k_wifi7_pci 0000:01:00.0: chip_id 0x2 chip_family 0x4 board_id 0xff soc_id 0x40170200
ath12k_wifi7_pci 0000:01:00.0: fw_version 0x110cffff ... WLAN.HMT.1.1.c5-00302
ath12k_wifi7_pci 0000:01:00.0: received scan start failure event     <-- over and over

wlan0 never associates, never gets an address.

The annoying part isn't that it happens, it's that you can't get out of it. I scripted the recovery attempts so they'd be consistent: 40s of waiting, then modprobe -r ath12k_wifi7 && modprobe ath12k_wifi7 and another 60s, then an rfkill block/unblock and another 60s. Nothing worked, six times out of six. Only a reboot fixed it, though it does sometimes heal itself after a later, longer suspend.

That last detail is what sent me looking at the driver, and it makes sense once you do. ath12k_pci_power_up() and ath12k_pci_power_down() (drivers/net/wireless/ath/ath12k/pci.c:1424-1470) only stop and start MHI and do a ath12k_pci_sw_reset(), which is a software reset. The real power sequence for this part hangs off qcom_pcie_host_deinit() -> pci_pwrctrl_power_off_devices() -> pwrseq_qcom_wcn_wlan_disable(), and there's no way to reach that from ath12k or from userspace. So the chip never actually gets power cycled, which is why a module reload does nothing, and why another suspend is the only thing that clears it.

Worth saying explicitly because it was my first instinct too: adding a "count consecutive scan failures and kick ab->reset_work" patch wouldn't help. ath12k_core_reset() goes through the same power_down/power_up pair, so it does what the module reload already did.

What triggers it

Short suspends, as far as I can tell:

suspend length	wedged
20s	6 of 6, then 2 of 3 on a rerun
150s	0 of 3

I want to be upfront that this isn't nailed down. The long run is only n=3, and I did all the short ones back to back, so run order and duration are tangled together. Fisher exact on the pooled numbers comes out around p=0.08, so it's suggestive and no more. I'm doing a finer interleaved sweep that logs the actual off-time rather than the requested one, and I'll follow up with that.

Things I checked and it isn't

I chased a couple of wrong ideas first, so to save anyone repeating them:

• Not the firmware. The blobs moved out of the device filesystem into extra-firmware recently (14ab5da, restored for SM8550/SM8650 in SM8550/SM8650: build the extra-firmware package #3112) so that was my first suspicion, but they're byte-for-byte identical to the 20260801 build I'd tested before, same WLAN.HMT.1.1.c5-00302 string.
• Not this PR's patches, as far as I can see. I looked hardest at 0502 (dropping IRQF_NO_SUSPEND on the IPCC mailbox) since it touches the pmic_glink path, but there's no route from IPCC to the WCN's rails, clocks or firmware, and RPMh has its own IRQ that the patch doesn't touch.
• Not sleep.sh rfkill-blocking the radio, which was my second theory. Setting wifi.enabled=0 makes the pre hook skip both wifictl pin and wifictl disable so the radio is left completely alone across the suspend, and it still wedged 2 of 3.
• Not any of the timeouts. This one took me a while: jiffies and CLOCK_MONOTONIC are frozen across suspend, and ath12k doesn't use boottime or the RTC anywhere, so none of its timeouts can even tell a 20s sleep from a 150s one. All those 20s constants in ath12k are a red herring.

Which leaves the chip's actual physical off-time as about the only thing left that scales with how long you slept. On that note, pwrseq_wcn7850_of_data in drivers/power/sequencing/pwrseq-qcom-wcn.c is the only WCN variant with no gpio_enable_delay_ms at all, and the helper that would enforce one measures from the last enable using jiffies, so it's inert across a system suspend regardless. I'm not sending a patch for that yet because I can't justify picking a delay value without better data.

How much does this matter

Honestly, probably not much day to day. Nobody suspends for 20 seconds on purpose, so what I'm doing is closer to a stress test than real use. But when it does happen there's no WiFi until you reboot, and the obvious way a handheld would stumble into it is an idle timer that suspends again a few seconds after a wake.

Two things that might be worth doing on the ROCKNIX side

Neither needs a kernel change:

1. A minimum gap between suspends. sleep.sh already writes /run/.last_sleep_time and nothing reads it, so the plumbing is sitting there. A sleep.d/pre guard that delays or refuses a suspend arriving a few seconds after the last resume would take the trigger away, and would cover any idle-based auto-suspend later on.

2. Making the pre hook wait. sleep.sh fires nohup wifictl disable and returns straight away, so whether ath12k has reached ATH12K_HW_STATE_OFF before the freezer runs is a race. That matters more than it looks, because ath12k_core_continue_suspend_resume() (core.c:102-116) only does anything when the radio is already down, so you end up taking one of two quite different paths depending on timing. Waiting for the rfkill block to land makes it deterministic.

Happy to send those as their own PR if you'd rather keep this one focused. Either way none of it blocks #2954 from where I'm sitting, deep suspend and UFS and the gamepad have been solid through every cycle I've thrown at it.

[shuuri-labs]

Following from the maintenance discussion on Discord: I think loki666b's concern is fair, and I don't have the bandwidth to keep carrying this against kernel bumps, so I'm closing this PR. The full stack lives on downstream in Pocknix, where all SM8550 devices have been running it for a while; anyone who wants to pick it up is welcome to everything here.

I've split out the one piece I do think belongs upstream: #3115, the IPCC wake mask - a small patch that's already applied to the SM8750.

Huge thanks to @jaewun for your wisdom and patches, and to @tiopex, @dcu, @Cyfarw9dd and @xiaodoudou for the device time :)

[loki666]

thanks to you

[dcu]

there's also ongoing work here

gh123man/armada-packages#1