Communitygithub.com

software-mansion/argent-device-interact

通过 argent MCP 工具与 iOS 模拟器、Android 模拟器或 Electron 应用进行交互。适用于点击 UI 元素、执行手势操作、滚动/滑动、输入文本、按硬件按钮、启动应用、打开 URL、截图或检查交互后的可见应用状态。

argent-device-interact 是什么?

该技能(software-mansion/argent-device-interact)基于 argent MCP(Model Context Protocol)工具,允许 AI 代理直接与移动或桌面模拟器/仿真器中的真实应用进行交互。它支持 iOS 模拟器、Android 模拟器和 Electron 应用,使 AI 能够执行点击、滑动、文本输入等操作,就像人类用户一样。主要功能包括:通过自然语言触发 UI 元素点击、手势(长按/双击)、滚动/滑动、硬件按钮按下(主屏幕/音量/电源)、启动应用、打开 URL、截图以及读取屏幕上的可见 UI 状态(文本、按钮、标签)。该技能对于需要视觉或行为验证的自动化任务特别有用,例如测试用户流程、检查布局变化或演示功能。它通过标准的 MCP 集成工作,能够在支持 MCP 的 IDE(如 VS Code、Cursor、Claude Code)中无缝使用,设计上高度模块化,易于通过配置文件扩展新平台或自定义交互。

兼容平台~Claude Code~Codex CLI~Cursor
npx skills add https://github.com/software-mansion/argent/tree/main/skills/argent-device-interact

在你喜欢的 AI 中提问

打开一个已预加载此 Agent Skill 的新对话。

文档

argent-device-interact 是做什么的?

使用 argent MCP 工具与 iOS 模拟器、Android 模拟器或 Electron 应用进行交互。适用于点击 UI 元素、执行手势操作、滚动/滑动、输入文本、按硬件按钮、启动应用、打开 URL、截图或检查交互后的可见应用状态。

功能

  • 点击 UI 元素:通过文本、标签或坐标识别元素并点击。
  • 执行手势:支持长按、双击、缩放等手势。
  • 滚动/滑动:在列表、页面或滚动视图中进行垂直或水平滚动。
  • 输入文本:向文本字段、搜索框等输入字符串。
  • 硬件按钮:模拟按下主屏幕、音量、电源等按钮。
  • 启动应用:通过 bundle ID 或包名启动应用。
  • 打开 URL:在模拟器/仿真器中打开指定 URL。
  • 截图:捕获当前屏幕状态。
  • 检查 UI 状态:读取当前可见的 UI 元素(如文本、按钮、标签)。

支持的平台

  • iOS 模拟器:通过 Xcode 集成。
  • Android 模拟器:通过 ADB 集成。
  • Electron 应用:通过 Electron 的辅助功能支持。

使用方法

在支持 MCP 的客户端(如 Claude Code、Cursor、VS Code)中配置该技能后,即可通过自然语言调用上述功能。

示例

  • “在 iOS 模拟器上点击‘登录’按钮”
  • “在 Android 模拟器中向下滚动列表”
  • “输入文本‘测试’到搜索框”
  • “按下主屏幕按钮”
  • “截图并保存”

配置

参考 argent MCP 工具的官方文档,在客户端的 mcp.json 或 settings.json 中添加该技能。

许可证

MIT

Individual skills in this repo

This repo contains 9 individual skills — each has its own dedicated page.

software-mansion/argent-android-emulator-setup

Set up and connect to an Android emulator using argent MCP tools. Use when starting a new session on Android, booting an emulator, getting a device serial, or before any UI interaction task.

software-mansion/argent-create-flow

Record a reusable flow (scripted sequence of MCP tool calls) that can be replayed later with a single command. Use when the user asks to create, record, or build a flow, or to script a sequence of device actions.

software-mansion/argent-ios-simulator-setup

Set up and connect to an iOS simulator using argent MCP tools. Use when starting a new session, booting an iOS simulator, getting an iOS UDID, or before any iOS simulator interaction task.

software-mansion/argent-metro-debugger

Debug a JS runtime via CDP using argent debugger tools. Primary path is React Native via Metro (iOS / Android); a subset of the tools (debugger-connect, debugger-status, debugger-evaluate, debugger-log-registry) also drive a Chromium (CDP) app's renderer (an Electron app, or any Chromium browser exposing CDP) through the same surface. Use when connecting to the runtime, inspecting React components, reading console logs, or evaluating JavaScript.

software-mansion/argent-native-profiler

Native profiling for CPU hotspots, UI hangs, memory issues. iOS via xctrace; Android via Perfetto. Use when diagnosing native-level performance issues.

software-mansion/argent-react-native-app-workflow

Step-by-step workflows for developing or debugging React Native apps on iOS simulator or Android emulator. Use when starting the app, debugging Metro, fixing builds, diagnosing runtime errors, or running tests.

software-mansion/argent-react-native-optimization

Optimizes a React Native app by profiling first to find real bottlenecks, then sweeping for mechanical issues. Entry-point for all performance work. Use when the app feels slow, user asks to optimize, fix re-renders, reduce jank, or improve startup. Delegates to argent-react-native-profiler for measurement.

software-mansion/argent-react-native-profiler

Profile a React Native Hermes app to measure re-render and CPU performance using argent profiler tools. Use when optimizing for performance, measuring before/after a fix, spotting slow components, diagnosing re-renders, checking CPU hotspots, or producing a ranked issue report.

software-mansion/argent-test-ui-flow

Autonomously test an app UI (iOS or Android) by running interact-screenshot-verify loops using argent MCP tools. Use when testing UI flows, verifying login works, testing navigation, running end-to-end UI test scenarios, manual QA steps, visible UI changes, or visual behavior.

相关技能