Usage Guide: Automating QA with AI

One of the primary use cases for the iOS Simulator MCP server is to empower AI assistants to perform Quality Assurance (QA) tasks. This allows for rapid validation of UI changes and behavior immediately after implementing new features.

By instructing your AI assistant within an MCP client like Cursor, you can automate interactions that would typically be manual, saving time and ensuring consistency.

Example Prompts for Common QA Tasks

Here are some examples of prompts you can give to your AI assistant to perform common QA checks.

Verifying UI Elements

To get a complete picture of all accessible elements on the current screen:

Verify all accessibility elements on the current screen

Testing Text Input

To ensure a text field is working correctly:

Enter "QA Test" into the text input field and confirm the input is correct

Checking Tap/Press Response

To simulate a user tapping a button or link at a specific location:

Tap on coordinates x=250, y=400 and verify the expected element is triggered

Validating Swipe Gestures

To test scrolling or other swipe-based navigation:

Swipe from x=150, y=600 to x=150, y=100 and confirm correct behavior

Inspecting a Specific Element

To get detailed information about an element at a certain point on the screen:

Describe the UI element at position x=300, y=350 to ensure proper labeling and functionality

Visual Inspection with AI

To let the AI 'see' the current state of the simulator:

View the current simulator screen

Capturing Media for Documentation

To save visual proof of a UI state or a bug:

Take a screenshot of the current simulator screen and save it to my_screenshot.png

To record a user flow or an animation:

Start recording a video of the simulator screen, then stop the recording when I'm done.

Advanced Workflow: A Full Test Case

The following is a complete, step-by-step test case taken from the project's QA.md file. It demonstrates how multiple tools can be chained together to perform a complex user flow test in the native iOS Photos app.

You can execute this test by giving these instructions sequentially to your AI assistant.

  1. Open the Photos app in the iOS simulator.

  2. Call get_booted_sim_id to get the UDID of the booted simulator.

  3. Call record_video to start a screen recording of the test.

  4. Call ui_describe_all to make sure we are on the "All Photos" tab.

  5. Call ui_describe_point to find the x and y coordinates for tapping the "Search" tab button.

  6. Call ui_tap to tap the "Search" tab button.

  7. Call ui_tap to focus on the Search text input.

  8. Call ui_type to type "Photos" into the Search text input.

  9. Call ui_describe_all to describe the page and find the first photo result.

  10. Call ui_describe_point to find the x and y coordinates for the first photo result's touchable area.

  11. Call ui_tap to tap the coordinates of the first photo result's touchable area.

  12. Call ui_swipe to swipe from the center of the screen down to dismiss the photo and go back to the "All Photos" tab.

  13. Call ui_describe_all to describe the page and verify we are back on the "All Photos" tab.

  14. Call screenshot to take a screenshot of the current page.

  15. Call ui_view to send the current page's image to the AI for visual confirmation.

  16. Call stop_recording to stop the screen recording and save the video file.