Input methods and accessibility

The keyboard is not part of your app. It is a different process, usually a different company's code, that connects to your text field through an interface the platform defines. Accessibility services use a parallel interface to read and act on the same field. A custom text editor that implements neither is invisible to both.

TL;DR#

• The input method framework connects an IME app to an editable text field through InputConnection.
• The IME is a separate app with its own window, its own process and its own lifecycle; it can be replaced.
• The accessibility framework builds a tree of nodes from the view hierarchy and exposes actions — click, scroll, set text — to services such as TalkBack.
• Both depend on the same view APIs: onCreateInputConnection for the IME and onInitializeAccessibilityNodeInfo for accessibility.
• A custom view that draws text itself and handles touches manually must implement both, or it is unusable for a large group of users.

Mental model#

Think of a translator between two people who share no language.

The text field speaks "characters, selection, composing regions". The keyboard speaks "keys, candidates, suggestions". The InputConnection is the interpreter: it translates in both directions and defines what each side may ask of the other.

An accessibility service is a second interpreter, standing beside the same conversation, describing it to someone who cannot see or touch the screen.

Where the analogy breaks#

A translator can improvise. InputConnection is a fixed protocol with a specific set of operations, and an IME that asks for something the protocol does not support gets a default answer rather than a negotiation. The same is true of accessibility: the tree of nodes is a lossy description of the view hierarchy, and what it leaves out is unavailable to the service.

The big picture#

The connection between the two systems is the view: it is the only place that knows both what is drawn and what is editable.

Deep dive#

The IME as an app#

An input method is a Service that declares an input method in its manifest. The system binds to it when a text field requests input, and the IME shows its own window — TYPE_INPUT_METHOD — in the IME band.

Because it is a separate app, it has a lifecycle, permissions and a process of its own. The user can install a different keyboard, and the system will use it for every app. That is a deliberate design choice: the keyboard sees everything you type, so it must be replaceable and auditable.

The InputConnection protocol#

When a view gains focus and is editable, the system asks the view for an InputConnection:

override fun onCreateInputConnection(outAttrs: EditorInfo): InputConnection {
    outAttrs.inputType = InputType.TYPE_CLASS_TEXT or InputType.TYPE_TEXT_FLAG_CAP_SENTENCES
    outAttrs.imeOptions = EditorInfo.IME_ACTION_SEND
    return super.onCreateInputConnection(outAttrs)
}

EditorInfo¹ describes what kind of input the field expects — text, number, email, password — and what the action button should say. The IME uses it to choose a layout and a label.

Once connected, the IME can:

• Commit text (commitText), set a composing region (setComposingText), delete text (deleteSurroundingText).
• Read text around the cursor (getTextBeforeCursor, getTextAfterCursor), which is how suggestions work.
• Move the cursor, select text, and send key events.

The protocol is asymmetric: the field can tell the IME about selection and text changes, but it cannot force the IME to do anything. That asymmetry is why IME behaviour varies between keyboards: the interesting parts — prediction, autocorrect, gesture typing — are entirely on the IME's side.

IME insets and layout#

The keyboard's window contributes an IME inset. An app that ignores it puts the focused field behind the keyboard, which is the single most common keyboard bug.²

The two levers are:

• android:windowSoftInputMode="adjustResize" — the window shrinks so the field stays visible. In edge-to-edge mode this is often the wrong choice, because the window already extends to the bottom.
• An inset listener or Modifier.imePadding() — the content is padded by the IME inset, so the field scrolls above the keyboard.

Full-screen IMEs and extract mode#

Some IMEs offer a full-screen editing mode for landscape, where the field is displayed inside the IME's own window. EditorInfo.IME_FLAG_NO_EXTRACT_UI disables it. This is one of the few places where an app can influence the IME's window rather than just its content.

The accessibility tree#

Accessibility services do not read pixels. They read a tree of AccessibilityNodeInfo objects, built from the view hierarchy at the service's request. For each view, the framework asks the view for its node info; the default implementation derives it from the view's text, content description, class name and enabled/clickable state.

That means accessibility is only as good as the view's metadata. A TextView is accessible by default; a custom view that draws text on a Canvas is not, because nothing tells the framework there is text there.

Making a custom view accessible#

override fun onInitializeAccessibilityNodeInfo(info: AccessibilityNodeInfo) {
    super.onInitializeAccessibilityNodeInfo(info)
    info.className = Button::class.java.name
    info.contentDescription = "Play"
    info.isClickable = true
    info.addAction(AccessibilityNodeInfo.AccessibilityAction.ACTION_CLICK)
}
 
override fun performAccessibilityAction(action: Int, args: Bundle?): Boolean {
    if (action == AccessibilityNodeInfo.ACTION_CLICK) {
        performClick()
        return true
    }
    return super.performAccessibilityAction(action, args)
}

The two overrides are the contract: describe yourself, and be able to act on the description. A view that does the first without the second is announced but unusable.

Focus, announcements, and live regions#

Accessibility services also depend on focus and announcements. View.announceForAccessibility tells the service that something changed. AccessibilityNodeInfo.isLiveRegion marks a node whose content changes should be announced automatically.

Compose exposes the same concepts through semantics: contentDescription, onClick, stateDescription and liveRegion.

Keyboard navigation and D-pad#

Hardware keyboards and TV remotes navigate by focus, not by touch. android:nextFocusDown and the focusable attributes control the order. A view that is not focusable is unreachable by a keyboard, even if it is perfectly usable by touch — the same class of gap as missing accessibility actions.

Why the IME protocol is so small#

A text editor needs rich behaviour: selection, composition, autocorrect, undo, formatting. InputConnection exposes almost none of it, because the IME is a separate app with a different UI and no access to your view tree.

The division of responsibility is deliberate. The app owns the text and the cursor; the IME owns the keyboard UI, the candidate list and the learning model. The protocol between them carries only the minimum the app needs to know: what changed, what is selected, what the field expects.

The consequence is that the interesting parts of a keyboard live entirely in the keyboard, which is why switching keyboards changes behaviour much more than the protocol would suggest.

Autofill is a third client#

Autofill services inspect the view hierarchy for fields, using the same EditorInfo and view metadata that the IME uses. A custom text field that does not expose the right autofillHints is invisible to autofill, exactly as it is invisible to accessibility when it does not expose node info.

This is the recurring pattern in this module: one view, several consumers, each reading the metadata the view publishes. A view that publishes nothing works only for the one consumer using raw touch.

Semantics in Compose#

Compose's accessibility model is built on semantics rather than on a view tree. A composable declares its meaning:

Modifier.semantics {
    contentDescription = "Play"
    role = Role.Button
    onClick { play(); true }
}

The semantics tree is what accessibility services read, and it is also what the test framework uses to find and act on elements. A composable with no semantics is as unreachable for a screen reader as a custom View with no node info.

What good keyboard support looks like#

Beyond the IME, an app should handle:

• Hardware keyboards: onKeyDown and focus order. A view that is not focusable cannot be reached.
• D-pad navigation: focus movement, not touch. TV apps live or die on this.
• IME actions: IME_ACTION_SEND, IME_ACTION_SEARCH, IME_ACTION_NEXT — the label and the behaviour must match, or the user presses a button that says "Send" and gets nothing.
• Enter/return behaviour: a multi-line field should not submit on Enter; a single-line field should.

None of these are optional for an app that expects to be usable by everyone.

Why the full-screen extract mode exists#

A landscape phone with a keyboard covering two-thirds of the screen leaves very little room for the field and its context. Extract mode gives the IME the whole screen, shows the field's text inside it, and lets the app keep editing.

It is a compromise that exists because the physical constraints are real. The app can disable it (IME_FLAG_NO_EXTRACT_UI) when its own layout already keeps the field visible, which is the better answer for most modern apps.

What an accessibility service can actually do#

A service with the right capability can:

• Read the node tree, including text, bounds and actions.
• Perform actions: click, scroll, set text, focus.
• Dispatch gestures through the accessibility framework rather than through touch.
• Track window changes and announcements.

That breadth is why accessibility services are a privileged capability, and why the platform treats them as a security-sensitive permission rather than a convenience.

Announcements and live regions#

Not every change is a focus change. A form validation error, a score update, a "message sent" toast — these need to be announced without moving focus.

View.announceForAccessibility pushes an announcement to the service. AccessibilityNodeInfo.isLiveRegion marks a node whose content changes should be announced automatically, which is the right tool for a status line that updates in place.

Compose's liveRegion semantics is the equivalent, and it is one of the few accessibility features that has no touch equivalent — there is nothing to see for a screen reader user, so the announcement is the entire experience.

Where custom views most often fail#

The pattern is consistent across custom controls:

1. No className — the service announces "view" rather than "button".
2. No content description — the control is announced as unlabelled.
3. No action — the control can be found but not activated.
4. No state — a checked toggle is announced identically whether checked or not.

Each is a few lines in onInitializeAccessibilityNodeInfo, and each is invisible to anyone testing with a finger.

How it evolved#

Under the hood#

• ‹›InputConnection.java↗the protocol between field and IME
• ‹›EditorInfo.java↗the field's description of what it expects
• ‹›AccessibilityNodeInfo.java↗the node model services read
• ‹›↗the accessibility service manager in system_server
• ‹›View.java↗onCreateInputConnection and onInitializeAccessibilityNodeInfo

Curiosity corner#

• The keyboard can read your password field's metadata but not its text — unless the field declares itself a password, in which case suggestions and learning are suppressed.
• Accessibility services can perform gestures. A service can dispatch a swipe or a tap through the accessibility framework rather than the touch pipeline, which is how switch access and voice control work.
• The IME band is above the app band but below the system band, which is why the keyboard covers your content but the navigation bar stays visible over it.

Try it#

$ adb shell uiautomator dump && adb shell cat /sdcard/window_dump.xml

$ adb shell dumpsys input_method | grep -i "mCurMethodId|mInputShown"

mCurMethodId=com.google.android.inputmethod.latin/com.android.inputmethod.latin.LatinIME

$ adb shell ime list -s

com.google.android.inputmethod.latin/.LatinIME

Check yourself#

Sources#