Skip to content

About TalkBack audits

An Android audit does not simulate a screen reader. It runs your app on a real device or emulator with the real TalkBack switched on. An agent operates the app the way a TalkBack user does. It swipes, touches points on the screen and double-taps, and it judges the app on what TalkBack says. This page covers how that works and why it is built that way.

Three parties take part in a run.

  • The device runs your app, TalkBack, and the Binclusive audit agent APK. The APK has two jobs. Its accessibility service carries out gestures on TalkBack’s cursor. It is also the device’s text-to-speech engine, so whatever TalkBack speaks passes through it and is written down as text.
  • The CLI on your machine talks to the device over adb. It sends each gesture to the agent, reads back the spoken text, and takes a screenshot.
  • The audit service decides what to do next. It asks the CLI for one action at a time and reads the result before it chooses the next.

This split is why the device needs the agent installed and set as the default text-to-speech engine. Without that, TalkBack would still speak but nothing would record it, and the audit would have nothing to judge.

A journey in the dashboard is plain text, such as “sign in and add an item to the cart”. The audit first turns it into steps a TalkBack user could follow, with each one tied to a screen and a gesture: “On the Login screen, activate ‘Sign in’”, not “Log in”. A journey that starts from a deep link opens that screen directly. Otherwise the walk starts from the app’s launch screen.

A tester then walks the steps in order. For each one it does what the step says, listens to every announcement along the way, and records a finding while TalkBack’s cursor is still on the element at fault, so the finding’s screenshot shows the right element. When a step is done it moves the plan on to the next one.

The tester judges the spoken phrase alone. The screenshot shows what a sighted user sees, and a gap between the two is the finding. A button drawn with a label that TalkBack announces only as “button” fails WCAG 4.1.2. The same goes for a change of state that is never spoken, or content that neither swiping nor touching reaches.

Before it records that something cannot be reached, the tester tries both ways a TalkBack user would. It swipes through the screen, then touches the spot where the control should be, such as the bottom edge for a navigation bar or the top for the app bar. Touching is ordinary TalkBack use when the control was already on screen. If the tester has to go looking for the result of its own action, such as a dialog it opened or an error it caused, the app should have moved focus there or announced it, and the tester records that as a finding.

The tester has a fixed set of actions, each one a gesture a TalkBack user makes:

  • move the cursor to the next or previous item, the one-finger swipe;
  • jump to the next heading or the next control, the two element types Android navigates by;
  • touch one named point on the screen and hear what is under it: the top, the centre, the bottom, or a corner;
  • double-tap the item the cursor is on;
  • type into the focused field;
  • press Back, Enter or Space, or return to the device launcher;
  • hear the last announcement again without moving.

Every action returns what TalkBack said and a screenshot with the focused item marked. The tester cannot tap a pixel it read off the screenshot or open a screen by name. If a user cannot reach something with these gestures, the tester cannot reach it either.

TalkBack adds its own words to what an app supplies. It appends usage hints such as “Double-tap to activate”, and it announces an item’s position, such as “Row 2” or “In list”. These are TalkBack’s narration, not part of your app’s accessible name.

Each finding is checked against the project’s earlier findings to decide whether it is new or one you already have. Before that check, the hint lines and position phrases are stripped from its evidence, and the stripped evidence is what the finding keeps. The finding is judged on the label, role and state your app provided, not on TalkBack’s narration around them.

TalkBack only moves its cursor when accessibility and touch exploration are on and its service is enabled. The run turns on those settings, enables the agent’s own accessibility service, and makes the agent the default text-to-speech engine. It records the value each one had before, and when the run ends it puts each back, including when you interrupt it. A setting that cannot be restored is reported with the command that restores it by hand.

The run also reads the settings back before it starts, and checks that Android actually connected both services. A setting can read back as “on” while Android never started the service behind it. TalkBack then grants no cursor, and a walk of the screen would report it as empty instead of failing. Checking first turns that silent empty result into a refusal you can act on.

To run an audit, see Run an Android audit with TalkBack.