Skip to content

[P4] Missing interactions: key events, alerts, long press, targeted input #40

Description

@docer1990

Problem

Common real-world interactions cannot be performed today:

  • Android: no key events — notably Enter / IME action after input_text, so keyboard-submitted forms cannot be completed. No clear-text-field. No long press / double tap.
  • iOS: no keyboard dismiss, no handling of system alerts (permission dialogs) — onboarding flows dead-end. No long press / double tap.
  • Both: input_text only types into the focused element; there is no "tap element X and type" one-shot. No pinch/zoom.

Scope (suggested tools)

  • android_press_key (keycode or named action: enter, tab, delete...)
  • android_clear_text
  • long_press / double_tap (both platforms, by coordinates or element)
  • ios_dismiss_keyboard
  • ios_handle_alert (accept/dismiss, optional button label)
  • input_text optional element selector (tap-then-type)
  • Pinch/zoom: nice-to-have, can be split out.

Acceptance criteria

  • New JSON-RPC methods in the respective automation servers with unit tests (Robolectric / XCTest where feasible).
  • MCP tools + CLI subcommands registered; EXPECTED_TOOLS + CLAUDE.md tables updated.

Priority: 4 (unblocks forms and onboarding flows).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions