For AI agents: the complete documentation index is available at /next/llms.txt, the full documentation bundle is available at /next/llms-full.txt, and this page is available as Markdown at /next/ai/agent-lynx/page-inspection.md.
AI logo
AI
  • English

      • Page Inspection and Interaction

      Once the agent has found a page, give it a task to complete. It uses snapshots to understand the page, chooses an element to act on, and reads the resulting state to verify the outcome.

      Switch a Theme and Verify the Result

      Continue the connection guide: the agent has loaded the lynx-devtool skill and found the Settings page. The conversation below uses the same client, android-phone:8901, and session, 1, to switch its theme.

      Example conversation5 tool calls
      You

      Switch that Settings page to the Ocean theme and check that it worked.

      Agent

      Approach:I will inspect the page, find the Ocean button, and record the current background color.

      agent-lynx snapshot --client android-phone:8901 --session 1Ocean button found
      Call
      # Inspect the page selected in the connection guide. This excerpt shows the theme controls.
agent-lynx snapshot --client android-phone:8901 --session 1
      Output
      @e1 [scroll-view] @(207,426) {scrollable}
└─ @e2 [view] @(207,436)
   ├─ @e3 [view] @(207,46)
   │  ├─ @e4 [text] "Settings" @(207,35)
   │  └─ …
   ├─ @e6 [view] @(207,153)
   │  ├─ …
   │  └─ @e10 [view] @(207,183)
   │     ├─ …
   │     └─ @e15 [view] @(324,182)
   │        └─ @e16 [text] "Ocean" @(323,183)
   └─ …
      agent-lynx get style @e1 --property background-color --client android-phone:8901 --session 1Current color recorded
      Call
      # Read the root container’s computed style to compare with the result after switching themes.
agent-lynx get style @e1 --property background-color --client android-phone:8901 --session 1
      Output
      background-color: rgba(9, 9, 11, 1)

      Approach:The Ocean label is inside the @e15 button. I will tap that container, then inspect the change.

      agent-lynx tap @e15 --snapshot --client android-phone:8901 --session 1Tapped; snapshot refreshed
      Call
      # Tap the Ocean button from the current snapshot and refresh the refs cached by the daemon.
agent-lynx tap @e15 --snapshot --client android-phone:8901 --session 1
      Output
      Tapped @e15 at (324,182). Snapshot refreshed with 69 refs.

      Approach:Next I will read the new snapshot, identify the page root again, and compare its background color.

      agent-lynx snapshot --client android-phone:8901 --session 1Fresh refs confirmed
      Call
      # Read the tree after the action. The theme change preserves its structure; this excerpt shows the same branch.
agent-lynx snapshot --client android-phone:8901 --session 1
      Output
      @e1 [scroll-view] @(207,426) {scrollable}
└─ @e2 [view] @(207,436)
   ├─ @e3 [view] @(207,46)
   │  ├─ @e4 [text] "Settings" @(207,35)
   │  └─ …
   ├─ @e6 [view] @(207,153)
   │  ├─ …
   │  └─ @e10 [view] @(207,183)
   │     ├─ …
   │     └─ @e15 [view] @(324,182)
   │        └─ @e16 [text] "Ocean" @(323,183)
   └─ …
      agent-lynx get style @e1 --property background-color --client android-phone:8901 --session 1Background color changed
      Call
      # Use the root ref from the new snapshot to check whether the background color changed.
agent-lynx get style @e1 --property background-color --client android-phone:8901 --session 1
      Output
      background-color: rgba(15, 23, 42, 1)

      Switched to Ocean. The page root's background color changed from rgba(9, 9, 11, 1) to rgba(15, 23, 42, 1), matching the Ocean background color in this example.

      The conversation uses the page structure to identify the target: the Ocean label is @e16, inside the @e15 button container. After tapping it, the agent reads a new tree and compares the root's computed style. Its conclusion comes from the page state before and after the action.

      An @eN ref identifies an element in the current page snapshot. The daemon retains these refs between calls. After navigation, reloads, or content changes, read a new snapshot and choose the target from that tree. This theme switch preserves the tree structure and its ref numbers; other changes can assign different numbers.

      Scroll to Inspect Offscreen Content

      A full snapshot can include nodes marked offscreen, helping the agent discover content beyond the viewport. To inspect the bottom of Settings, it first finds the scrollable container, scrolls, and then reads the visible content.

      Example conversation3 tool calls
      You

      Now check what else is at the bottom of this page. Just look; don’t tap any buttons.

      Agent

      Approach:I will inspect the full snapshot to find the scroll container and the content outside the viewport.

      agent-lynx snapshot --client android-phone:8901 --session 1Bottom buttons are offscreen
      Call
      # The full snapshot includes offscreen nodes. This excerpt shows the bottom of the page.
agent-lynx snapshot --client android-phone:8901 --session 1
      Output
      @e1 [scroll-view] @(207,426) {scrollable}
└─ @e2 [view] @(207,436)
   ├─ …
   └─ @e61 [view] @(207,816)
      ├─ @e62 [view] @(207,818)
      │  ├─ @e63 [text] "Danger Zone" @(207,808)
      │  └─ @e64 [text] "Irreversible and destructive actions." @(207,832)
      └─ @e65 [view] @(207,871) {offscreen}
         ├─ @e66 [view] @(91,871) {offscreen}
         │  └─ @e67 [text] "Delete Account" @(91,871) {offscreen}
         └─ @e68 [view] @(186,871) {offscreen}
            └─ @e69 [text] "Cancel" @(186,871) {offscreen}

      Approach:The bottom nodes are marked offscreen. I will scroll @e1 down, then inspect the visible content.

      agent-lynx scroll @e1 --direction down --snapshot --client android-phone:8901 --session 1Scrolled down
      Call
      # Scroll the container marked scrollable in the latest snapshot, then refresh the refs.
agent-lynx scroll @e1 --direction down --snapshot --client android-phone:8901 --session 1
      Output
      Scrolled @e1 down. Snapshot refreshed with 69 refs.
      agent-lynx snapshot --visible-only --client android-phone:8901 --session 1Visible content confirmed
      Call
      # Read the visible elements after scrolling. This excerpt shows the bottom nodes now inside the viewport.
agent-lynx snapshot --visible-only --client android-phone:8901 --session 1
      Output
      @e1 [scroll-view] @(207,426) {scrollable}
└─ @e2 [view] @(207,376)
   ├─ …
   └─ @e61 [view] @(207,756)
      ├─ @e62 [view] @(207,758)
      │  ├─ @e63 [text] "Danger Zone" @(207,748)
      │  └─ @e64 [text] "Irreversible and destructive actions." @(207,772)
      └─ @e65 [view] @(207,811)
         ├─ @e66 [view] @(91,811)
         │  └─ @e67 [text] "Delete Account" @(91,811)
         └─ @e68 [view] @(186,811)
            └─ @e69 [text] "Cancel" @(186,811)

      The bottom of the page contains Danger Zone, with Delete Account and Cancel buttons and a description of irreversible actions. I only scrolled to inspect the content.

      The visible snapshot after scrolling confirms that the bottom buttons have entered the viewport. Use snapshot --visible-only to inspect the current screen, or a full snapshot to discover content outside it.

      Inspection and Interaction Capabilities

      CapabilityWhat it supports
      Inspect page structuresnapshot exposes the element tree, text, element refs, and flags such as editable and scrollable. It can include offscreen content or show only visible elements.
      Read text and stylesget text reads element text; get style reads computed styles, providing evidence of content, color, and size changes.
      Tap and long-presstap and long-press act on an element ref to trigger controls and gestures such as opening a context menu.
      Edit inputfill replaces an editable field's contents, and clear empties it. The snapshot's editable flag identifies supported targets.
      Scroll contentscroll moves a specified scroll container up, down, left, or right to bring offscreen content into view.
      Wait for page changeswait polls fresh snapshots until matching text is found or an element ref resolves, with a configurable timeout.

      The next guide's annotated screenshot uses the same Settings example to connect visible boxes with element refs. If an action does not produce the expected change, continue with logs and runtime debugging. See the CLI reference for parameters, snapshot refresh behavior, and wait units.

      Except as otherwise noted, this work is licensed under a Creative Commons Attribution 4.0 International License, and code samples are licensed under the Apache License 2.0.