如何使用无障碍服务“为用户采取行动”？

Question

背景

返回few years ago，我询问 TeamViewer 如何让用户在不与设备进行正常交互的情况下控制设备。有人告诉我，这是制造商专门为此应用程序允许的特殊“后门”，并且只能对其他应用程序使用 root 权限。

看到像"Airplane Mode Shortcut" 这样的应用程序允许通过自动导航到其屏幕并切换开关来切换飞行模式，这让我意识到这种情况已经改变了。

问题

据说在the docs：

从 Android 4.0（API 级别 14）开始，无障碍服务可以 代表用户行事，包括更改输入焦点和 选择（激活）用户界面元素。在 Android 4.1 (API 第 16 级）操作范围已扩大到包括滚动 列表并与文本字段交互。无障碍服务可以 还可以采取全局操作，例如导航到主屏幕， 按返回按钮，打开通知屏幕和最近 应用程序列表。 Android 4.1 还包括一种新型的焦点， Accessibilty Focus，使所有可见元素都可以通过 无障碍服务。

这些新功能使开发人员能够 无障碍服务以创建替代导航模式，例如 手势导航，并为残障用户提供更好的控制 他们的 Android 设备。

但是没有关于如何使用它的更多信息。 只有我找到的示例位于底部，但这些示例非常古老，并且是 apiDemos 捆绑包的一部分。

问题

如何制作一个可以查询、聚焦、点击、输入文本以及执行其他 UI 相关操作的服务？

Answer 1

通过实现AccessibilityService (https://developer.android.com/training/accessibility/service.html)，您可以访问这些功能。

您可以检查或对用户最后交互的元素执行操作，也可以检查当前处于活动状态的整个应用程序。

通过实现onAccessibilityEvent(AccessibilityEvent event) 拦截用户事件，在这里您可以使用event.getSource() 检索虚拟视图（表示原始视图），然后使用getClassName() 或getText() 或您在文档中找到的任何内容进行检查。

通过调用getRootInActiveWindow() 检查整​​个应用程序并使用getRootInActiveWindow().getChild(index) 遍历虚拟视图树。

getRootInActiveWindow() 和event.getSource() 都返回AccessibilityNodeInfo，您可以在其上调用performAction(action) 并执行单击、设置文本等操作。

示例：Play 商店

打开 Play 商店应用后，搜索“facebook”应用并在 Play 商店中打开它的页面。

    @Override
    public void onAccessibilityEvent(final AccessibilityEvent event) {

        AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
        //Inspect app elements if ready
        if (rootInActiveWindow != null) {
            //Search bar is covered with textview which need to be clicked
            List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
            if (searchBarIdle.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
                searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
            }
            //Check is search bar is visible
            List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
            if (searchBars.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBars.get(0);
                //Check is searchbar have the required text, if not set the text
                if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
                    Bundle args = new Bundle();
                    args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
                    searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
                } else {
                    //There is no way to press Enter to perform search, so find corresponding suggestion and click
                    List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
                    for (AccessibilityNodeInfo suggestion : searchSuggestions) {
                        if(suggestion.getText().toString().equals("Facebook")) {
                            //We found textview, but its not clickable, so we should perform the click on the parent
                            AccessibilityNodeInfo clickableParent = suggestion.getParent();
                            clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                        }
                    }
                }
            }


        }
   }

---

编辑：下面的完整代码：

MyAccessibilityService

public class MyAccessibilityService extends AccessibilityService {

    @Override
    public void onCreate() {
        super.onCreate();
        Log.d("MyAccessibilityService", "onCreate");
    }

    @Override
    public void onAccessibilityEvent(final AccessibilityEvent event) {
        Log.d("MyAccessibilityService", "onAccessibilityEvent");
        AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
        //Inspect app elements if ready
        if (rootInActiveWindow != null) {
            //Search bar is covered with textview which need to be clicked
            List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
            if (searchBarIdle.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
                searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
            }
            //Check is search bar is visible
            List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
            if (searchBars.size() > 0) {
                AccessibilityNodeInfo searchBar = searchBars.get(0);
                //Check is searchbar have the required text, if not set the text
                if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
                    Bundle args = new Bundle();
                    args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
                    searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
                } else {
                    //There is no way to press Enter to perform search, so find corresponding suggestion and click
                    List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
                    for (AccessibilityNodeInfo suggestion : searchSuggestions) {
                        if (suggestion.getText().toString().equals("Facebook")) {
                            //We found textview, but its not clickable, so we should perform the click on the parent
                            AccessibilityNodeInfo clickableParent = suggestion.getParent();
                            clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                        }
                    }
                }
            }
        }
    }

    @Override
    public void onInterrupt() {
    }
}

AndroidManifest.xml

<manifest xmlns:android="http://schemas.android.com/apk/res/android"
          package="com.example.findfacebookapp">

    <application
        android:allowBackup="true"
        android:icon="@mipmap/ic_launcher"
        android:label="@string/app_name"
        android:supportsRtl="true"
        android:theme="@style/AppTheme">
        <activity android:name=".MainActivity">
            <intent-filter>
                <action android:name="android.intent.action.MAIN"/>

                <category android:name="android.intent.category.LAUNCHER"/>
            </intent-filter>
        </activity>
        <service
            android:name=".MyAccessibilityService"
            android:label="@string/accessibility_service_label"
            android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
            <intent-filter>
                <action android:name="android.accessibilityservice.AccessibilityService"/>
            </intent-filter>

            <meta-data
                android:name="android.accessibilityservice"
                android:resource="@xml/accessibility_service_config"/>
        </service>
    </application>

</manifest>

res/xml/accessibility_service_config.xml

<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes="typeAllMask"
    android:accessibilityFeedbackType="feedbackAllMask"
    android:accessibilityFlags="flagDefault"
    android:canRequestEnhancedWebAccessibility="true"
    android:canRetrieveWindowContent="true"
    android:description="@string/app_name"
    android:notificationTimeout="100"/>

MainActivity

public class MainActivity extends AppCompatActivity {

    public void onEnableAccClick(View view) {
        startActivityForResult(new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS), 1);
    }

}