Testing

A JVM test runs in milliseconds. A device test takes seconds to install, launch and synchronise.¹ Everything about Android's recommended architecture — ViewModels, repositories, interfaces — exists partly to move as much verification as possible into the fast column.

TL;DR#

• Unit tests (JVM) are fast and cover ViewModels, repositories and mappers. They are the bulk of a healthy suite.¹
• Robolectric (glossary definition) runs Android framework code on the JVM, for the cases where a fake is not enough.
• Instrumented tests (androidTest) run on a device or emulator and cover integration: database, navigation, UI.
• Compose UI tests assert on semantics, and are the cheapest way to test a screen's states.
• Macrobenchmark (glossary definition) covers performance; baseline profiles make the measured build representative.

Mental model#

A health check-up with three tiers.

A blood test at home (unit test) is cheap, fast and catches most problems. A visit to a clinic (Robolectric) is slower but covers things the home test cannot. A full scan (instrumented) is expensive and reserved for what only it can see.

Where the analogy breaks#

The tiers are not ordered by importance; they are ordered by cost. A screen that only an instrumented test can verify is a screen whose architecture is hiding something — usually that the logic lives in the view.

The big picture#

Deep dive#

Unit tests: the bulk#

class NotesViewModelTest {
 
    private val repository = FakeNotesRepository()
    private val dispatcher = UnconfinedTestDispatcher()
    private val viewModel = NotesViewModel(repository, dispatcher)
 
    @Test
    fun `empty repository produces empty state`() = runTest {
        viewModel.state.test {
            assertEquals(NotesUiState(), awaitItem())
        }
    }
 
    @Test
    fun `refresh failure surfaces an error`() = runTest {
        repository.failWith = IOException("offline")
 
        viewModel.refresh()
        advanceUntilIdle()
 
        assertEquals("offline", viewModel.state.value.error)
    }
}

Four things make that test possible, and each is an architectural decision:

1. NotesRepository is an interface, so a fake can implement it.
2. The dispatcher is a constructor parameter, so the test can use a test dispatcher.
3. The state is a value, so assertions are assertEquals.
4. The ViewModel has no Context, so no Robolectric is needed.

runTest and advanceUntilIdle come from kotlinx-coroutines-test.¹ UnconfinedTestDispatcher runs coroutines eagerly, which removes the need for advanceUntilIdle in most cases and makes tests deterministic.

Fakes versus mocks#

class FakeNotesRepository : NotesRepository {
    private val notes = MutableStateFlow<List<Note>>(emptyList())
    var failWith: Throwable? = null
 
    override fun observe(query: String): Flow<List<Note>> =
        notes.map { list -> list.filter { it.title.contains(query, ignoreCase = true) } }
 
    override suspend fun refresh(): Result<Unit> {
        failWith?.let { return Result.failure(it) }
        return Result.success(Unit)
    }
 
    fun emit(vararg items: Note) { notes.value = items.toList() }
}

A hand-written fake is usually better than a mocking framework on Android:

• It has behaviour, not just recorded calls, so a test asserting "state becomes Ready" works.
• It compiles against the interface, so an interface change breaks the fake immediately.
• It is reusable across tests, which is how a test suite stays readable.

Mocking frameworks are still useful for verifying an interaction ("was analytics.track called with these parameters?"). Use them for that, not for state.

Robolectric: the middle tier#

When the code genuinely needs framework classes — a Context, a Resources, a Looper — Robolectric runs a simulated Android runtime on the JVM.

@RunWith(RobolectricTestRunner::class)
class NotesFormatterTest {
 
    @Test
    fun `formats a relative timestamp`() {
        val context = ApplicationProvider.getApplicationContext<Context>()
        val formatter = NotesFormatter(context)
 
        val label = formatter.relative(Instant.now().minus(2, ChronoUnit.HOURS))
 
        assertEquals("2 hours ago", label)
    }
}

Robolectric is slower than a plain unit test (hundreds of milliseconds versus single-digit milliseconds) but far faster than an emulator. It is the right tool for resource lookups, Context wrappers, and framework behaviour that a fake cannot reproduce.

Compose UI tests#

Compose UI tests do not need a device if you use Robolectric, and they assert on semantics rather than on pixels:¹

class NotesScreenTest {
 
    @get:Rule val rule = createComposeRule()
 
    @Test
    fun emptyState_showsMessage() {
        rule.setContent {
            NotesScreen(state = NotesUiState(), onQueryChange = {}, onDelete = {})
        }
 
        rule.onNodeWithText("No notes yet").assertIsDisplayed()
    }
 
    @Test
    fun deleteButton_invokesCallback() {
        var deleted: Note? = null
        val note = Note("1", "Shopping", Instant.EPOCH)
 
        rule.setContent {
            NotesScreen(
                state = NotesUiState(notes = persistentListOf(note)),
                onQueryChange = {},
                onDelete = { deleted = it },
            )
        }
 
        rule.onNodeWithContentDescription("Delete note").performClick()
 
        assertEquals(note, deleted)
    }
}

The second test is the payoff for a stateless screen: no ViewModel, no repository, no device — just a callback assertion. That is what "hoist state" buys you in test terms.

Instrumented tests: the small top#

Some things genuinely require a device:

• Room schema and migrations.
• Navigation graph behaviour.
• Cross-process interactions.
• Anything that depends on the real Looper and real timing.

@RunWith(AndroidJUnit4::class)
class NotesDaoTest {
 
    private lateinit var db: NotesDatabase
    private lateinit var dao: NotesDao
 
    @Before fun setUp() {
        db = Room.inMemoryDatabaseBuilder(
            ApplicationProvider.getApplicationContext(),
            NotesDatabase::class.java,
        ).build()
        dao = db.notesDao()
    }
 
    @After fun tearDown() = db.close()
 
    @Test fun upsert_replacesExistingRow() = runTest {
        dao.upsertAll(listOf(NoteEntity("1", "A", 0)))
        dao.upsertAll(listOf(NoteEntity("1", "B", 1)))
 
        assertEquals("B", dao.observeMatching("").first().single().title)
    }
}

An in-memory database is fast and isolated; it is still an instrumented test because Room's generated code and SQLite are real.

Testing migrations#

A migration test is the highest-value instrumented test in most apps, because a broken migration is a data-loss bug that no unit test catches.

@RunWith(AndroidJUnit4::class)
class MigrationTest {
    @get:Rule
    val helper = MigrationTestHelper(
        InstrumentationRegistry.getInstrumentation(),
        NotesDatabase::class.java,
    )
 
    @Test
    fun migrate1To2_preservesRows() {
        helper.createDatabase(TEST_DB, 1).apply {
            execSQL("INSERT INTO notes (id, title) VALUES ('1', 'A')")
            close()
        }
 
        val db = helper.runMigrationsAndValidate(TEST_DB, 2, true, MIGRATION_1_2)
 
        db.query("SELECT title FROM notes").use { cursor ->
            assertTrue(cursor.moveToFirst())
            assertEquals("A", cursor.getString(0))
        }
    }
}

Performance tests#

Macrobenchmark measures what users feel: startup and frame timing, on a release build.²

@get:Rule val benchmarkRule = MacrobenchmarkRule()
 
@Test
fun scroll_isNotJanky() = benchmarkRule.measureRepeated(
    packageName = "com.example.notes",
    metrics = listOf(FrameTimingMetric()),
    iterations = 5,
    startupMode = StartupMode.WARM,
) {
    startActivityAndWait()
    device.findObject(By.res("notes-list")).fling(Direction.DOWN)
}

The result is a FrameTimingMetric with percentiles. Comparing two runs of the same test with a baseline profile installed versus not is the standard way to prove a performance change.

Test doubles for the framework#

Some framework classes are hard to fake because they are final or because they are constructed by the system. Three strategies, in order of preference:

Wrap them in an interface you own. A Clock, a DispatcherProvider, a FileStore. The wrapper is trivial and the fake is trivial.

Use a library fake. TestNavHostController for navigation, TestDriver for WorkManager, an in-memory Room database for persistence. These are the real implementation with the device removed.

Use Robolectric. When the code genuinely needs a Context, a Resources or a Looper, Robolectric provides a simulated runtime. It is the last resort because it adds seconds to the suite.

The ordering is not arbitrary. Each step down costs more per test, and the cost multiplies across a suite. A team that reaches for Robolectric first usually has a codebase that constructs its dependencies internally.

Keeping the suite fast#

A test suite that takes twenty minutes stops being run. Three practices keep it in the seconds:

• One assertion per behaviour, not one test per method. A test that checks four things fails with an ambiguous message.
• No sleeps. advanceUntilIdle() and advanceTimeBy() are deterministic; Thread.sleep(1000) is not.
• Parallel modules. Gradle runs independent modules' tests in parallel; a monolith cannot.

// Deterministic and instant, even though it tests a 500 ms debounce.
@Test
fun `debounce collapses rapid input`() = runTest {
    val viewModel = SearchViewModel(repository, testDispatcher)
    viewModel.onQueryChange("a")
    viewModel.onQueryChange("ab")
    advanceTimeBy(300)
    assertEquals(1, repository.calls)
}

What to test, and what not to#

Test:

• State transitions. Given this input, this state.
• Mapping. DTO → entity → domain, including nulls and malformed dates.
• Migrations. Data survives version changes.
• Business rules. The logic that would be expensive to get wrong.
• Critical user paths. One or two end-to-end tests, not fifty.

Do not test:

• Framework behaviour. That TextView renders text is Google's test, not yours.
• Generated code. Room's DAO implementation, Hilt's graph, Compose's compiler output.
• Trivial getters. They add suite runtime and catch nothing.
• Implementation details. A test that asserts a private method was called will break on every refactor and catch no bugs.

Flakiness is a design signal#

A test that fails intermittently is usually reporting a real property of the code: an un-injected clock, a shared mutable singleton, a dispatcher that was not substituted, or an ordering assumption.

The wrong fix is a retry. The right fix is to find which of those four it is — and each of them is also a production bug waiting to happen. A test that depends on wall-clock time will behave differently on a device whose clock has drifted; a test that depends on a singleton will behave differently when the singleton is initialised in a different order.

That is why "the test is flaky" is worth taking seriously rather than quarantining. It is the cheapest available detector for a category of bug that is otherwise invisible until a user reports it.

A suite that ships#

The practical target is not a coverage number; it is a suite that runs in under a minute and that the team trusts enough to run before every commit. Three properties get you there:

• Fast by default. JVM tests for everything that does not need a device, and the device tests kept few and deliberate.
• Deterministic. No sleeps, no real dispatchers, no wall-clock time.
• Diagnostic. A failure names the behaviour that broke, not the line that changed.

A suite with those three properties gets run. A suite without them gets skipped, and a skipped suite has the same value as no suite at all.

Coverage as a diagnostic, not a target#

Line coverage tells you what ran, not what was verified. A suite with 90% coverage that asserts nothing is worse than 40% coverage with real assertions, because it creates false confidence.

Use coverage to find untested branches in complex code — a migration with no test, an error path that never runs. Do not use it as a gate that incentivises assertion-free tests.

How it evolved#

The tooling moved in one direction: make tests faster and more representative. Robolectric made framework code testable on the JVM, Compose made UI testable without a device, Macrobenchmark made performance measurable in CI, and baseline profiles made the measured build match what users run.

Under the hood#

• ‹›Instrumentation.java↗the entry point instrumented tests run through
• ‹›Looper.java↗the loop a device test must synchronise with
• ‹›View.java↗what Espresso drives and what Compose no longer needs

Curiosity corner#

• Espresso's idling resources exist because the main thread's queue can be non-empty when a test asserts. The modern equivalent in Compose is the test clock, which is deterministic by construction.
• runTest uses a virtual clock. A test that waits an hour takes microseconds, which is why testing a debounce is cheap.
• A migration test is the only test that catches data loss, because it exercises the real SQLite upgrade path.

Try it#

$ 
./gradlew :app:testDebugUnitTest

$ 
./gradlew :app:connectedDebugAndroidTest

Check yourself#

Sources#