API: UI Inspection
Tools for inspecting the user interface of the running application by querying accessibility information and capturing the screen content.
ui_describe_all
Describes accessibility information for the entire screen in the iOS Simulator.
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
udid |
string |
No | UDID of the target simulator. If omitted, the booted simulator is used. |
The udid must match the UUID format: ^[0-9A-Fa-f]{8}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{4}-[0-9A-Fa-f]{12}$
Returns
A JSON string containing a nested tree of all accessibility elements on the screen.
ui_describe_point
Returns the accessibility element at a given (x, y) coordinate on the iOS Simulator's screen.
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
udid |
string |
No | UDID of the target simulator. If omitted, the booted simulator is used. |
x |
number |
Yes | The x-coordinate of the point to inspect. |
y |
number |
Yes | The y-coordinate of the point to inspect. |
Returns
A JSON string containing the accessibility information for the element found at the specified point.
ui_view
Captures a compressed screenshot of the current simulator view and returns it as image content for an AI to see.
Parameters
| Name | Type | Required | Description |
|---|---|---|---|
udid |
string |
No | UDID of the target simulator. If omitted, the booted simulator is used. |
Returns
An image content block containing the base64-encoded JPEG data of the screenshot, along with a confirmation text message.