✨ Add CLI-based computer use for Pi on Hyprland

[davidgasquez]

Enable Pi to inspect and control the native Hyprland desktop through CLI commands + a small skill, keeping the existing agent-browser workflow for websites.

Proposed approach

• wdotool for input, grim for screenshots, and hyprctl for window/workspace state and control; add a thin wrapper only where it simplifies coordinates or verification.
• Store the skill in agents/skills/ using the existing symlink setup. Pi can execute commands through Bash and read screenshots; no Pi extension should be necessary.

Local constraints
Custom Arch + Hyprland Lua configuration, Spanish keyboard (es), and Zephyr displays at 5/3 scale, including HDMI-A-1 at 3840×2160. wtype, hyprshot, and agent-browser are already declared; verify/add explicit dependencies. pi-sandbox creates a temporary working directory, not desktop isolation.

Acceptance

• Capture → inspect → act → verify works with Unicode text, shortcuts, clicks, scrolling, and window focus.
• Explicit monitor targeting and screenshot/pointer coordinate conversion work at fractional scale and nonzero monitor origins.
• Use Lua-compatible dispatchers and the native Wayland backend; verify results on the live desktop before calling the setup complete.

Research / references

• wdotool: native Wayland CLI; output-targeted pointer support was fixed in 0.5.0.
• Hetraeus dotfiles: exact-stack precedent, including Lua dispatchers; run-or-raise wrapper.
• Hdprajwal skill pack: useful observe/verify structure; replace older dispatch syntax and Bash integer arithmetic that cannot handle fractional scales.
• Castrozan desktop skill: concise skill + shell wrappers; its ydotool mouse commands need correction/replacement.
• Kabilan's acu CLI + skill: good single-command design, but Niri-specific.

No verified ready-made skill for the exact trio was found. Hypruse is MCP-oriented; Cua Driver offers a full CLI but requires a daemon and has less-established Hyprland validation. Research was source inspection, not live desktop testing.

[IlyasKhallouki]

@davidgasquez Author of hypruse here. Fair call on "MCP-oriented": as of 0.11.0 it's also a plain CLI plus a skill, which is what you're describing. hypruse desktop, hypruse click_ui Save --window 0x…, hypruse keyboard type "…" --window 0x…, hypruse zoom X Y; compact text output, exit codes for refusals, a file path for captures. hypruse skill install drops a SKILL.md into ~/.pi/agent/skills and ~/.agents/skills (or npx skills add IlyasKhallouki/hypruse). Your constraints are already covered: Lua-config dispatchers, unicode typing via wtype's real keymap, and one global coordinate space that handles fractional scale and nonzero monitor origins. Happy to fix whatever doesn't fit on your setup.

On a separate note, Thank you for giving hypruse a chance! It's been very surreal seeing great engineers like test my app or even drive it.