App data and databases

An app's data has a lifecycle longer than its process and sometimes longer than its installation. Choosing where to put it determines what survives a crash, what survives an uninstall, what syncs to a new phone, and what the user can see in a file manager.

TL;DR#

• Internal files are private, backed up (unless excluded), and deleted on uninstall.
• SharedPreferences is a simple key-value store with a synchronous commit and an asynchronous apply; it is still fine for small settings and is being replaced by DataStore for new code.
• SQLite is the embedded relational database; Room is the AndroidX layer that gives it compile-time checks.
• Cache directories are evictable — the system may delete from them, so nothing essential belongs there.
• Auto Backup covers most app data by default; allowBackup and the backup rules control what is included.

Mental model#

Think of a workshop with several kinds of storage.

The toolbelt is memory: fast, small, gone when you leave.

The bench drawer is SharedPreferences or DataStore: small items you need quickly, labelled and easy to reach.

The filing cabinet is SQLite: structured records you query.

The scrap bin is the cache directory: useful while you are working, emptied without notice.

The shipping crate is backup: what leaves the workshop and arrives at the next one.

Where the analogy breaks#

A workshop's drawers do not have concurrency rules. SharedPreferences is safe to read from multiple threads but its writes are asynchronous, DataStore serialises writes through a coroutine, and SQLite has its own locking model. The choice between them is as much about concurrency as about size.

The big picture#

The treemap's groups differ less in size than in rules. Two directories of the same size can have completely different survival characteristics.

Deep dive#

Internal files#

/data/user/0/<package>/files/

Private, readable only by the app's UID, deleted on uninstall. This is where user-generated content that belongs to the app should live.

val file = File(context.filesDir, "notes.json")
file.writeText(json)

For larger or structured data, openFileOutput and openFileInput are the classic APIs; for streaming, a FileInputStream is simpler.

SharedPreferences#

A key-value store backed by an XML file, loaded into memory on first access.

val prefs = context.getSharedPreferences("settings", Context.MODE_PRIVATE)
prefs.edit().putBoolean("dark_mode", true).apply()

Two behaviours matter:

• apply() is asynchronous. It writes to memory immediately and to disk in the background. commit() is synchronous and returns whether the write succeeded. Most code uses apply().
• The whole file is rewritten on every change. That is fine for a handful of settings and wasteful for a large map.

SharedPreferences is not deprecated, but new code is encouraged to use DataStore.

DataStore#

DataStore is the AndroidX replacement. It has two flavours: Preferences DataStore for key-value data and Proto DataStore for typed messages.

val Context.settings: DataStore<Preferences> by preferencesDataStore(name = "settings")
 
val darkMode: Flow<Boolean> = context.settings.data
    .map { it[darkModeKey] ?: false }

The differences that matter:

• Asynchronous by design, exposed as a Flow.
• Transactional, with a serialised write path, so concurrent writers cannot interleave.
• No partial writes on the main thread.
• Corruption handling via a ReplaceFileCorruptionHandler.

SQLite and Room#

SQLite is a relational database embedded in the platform. It is a library, not a server: the database is a file, and the app's process reads and writes it directly.

Room is the AndroidX layer over SQLite. It generates code from annotated classes:

@Entity
data class Note(
    @PrimaryKey(autoGenerate = true) val id: Long = 0,
    val title: String,
    val body: String,
)
 
@Dao
interface NoteDao {
    @Query("SELECT * FROM note ORDER BY id DESC")
    fun observeAll(): Flow<List<Note>>
 
    @Insert
    suspend fun insert(note: Note): Long
}

The generated code gives compile-time SQL verification, type-safe queries, and coroutine support. The alternative — raw SQLiteOpenHelper and Cursor — is still supported and still appropriate for a trivial database.

Migrations#

A database schema changes. The two options are destructive migration (drop and recreate, losing data) and a proper migration:

val MIGRATION_1_2 = object : Migration(1, 2) {
    override fun migrate(db: SupportSQLiteDatabase) {
        db.execSQL("ALTER TABLE note ADD COLUMN pinned INTEGER NOT NULL DEFAULT 0")
    }
}

Room validates the schema at open time. If the migration produces a schema that does not match the entity, it throws rather than silently corrupting data — which is the main reason to use Room for anything nontrivial.

Cache directories#

/data/user/0/<package>/cache/
/storage/emulated/0/Android/data/<package>/cache/

The system may delete from these directories when storage is low. They are for data the app can regenerate: decoded images, downloaded thumbnails, temporary exports.

A file that the user would be upset to lose does not belong in a cache directory. This is one of the most common data-loss bugs in apps that download content.

Backup#

Auto Backup, introduced in API 23·Android 6.0, uploads app data to the user's cloud backup.¹ It includes most files and databases by default, up to a quota, and excludes caches.

Control it with:

<application
    android:allowBackup="true"
    android:fullBackupContent="@xml/backup_rules"
    android:dataExtractionRules="@xml/data_extraction_rules" />

dataExtractionRules (for API 31·Android 12 and later) covers both cloud backup and device-to-device transfer. That distinction matters: a user moving to a new phone gets a different mechanism from a restore after a factory reset, and the rules can differ.

Files the user should see#

If a file must be visible in a file manager or to other apps, it belongs in MediaStore (if it is media) or in a user-chosen SAF tree (if it is a document). Writing it to the app's external directory makes it visible over USB but not to other apps, which is usually the wrong compromise.

Encryption#

Files are encrypted at rest by the platform's file-based encryption, keyed to the user's credential. An app does not need to encrypt its own files for the at-rest threat model — but it may still want to for defence in depth, or for data it exports outside the sandbox.

Choosing between DataStore and Room#

Both store structured data; they solve different shapes of problem.

DataStore is for a small, whole-value document: settings, a session token, a last-known position. It reads and writes the entire value, and its API is a Flow of the current value.

Room is for a collection of records you query, filter and join. It gives you SQL, indexes and migrations.

The mistake in both directions is common: a DataStore file with a thousand entries (a database in disguise) or a Room table with one row (a file in disguise).

Transactions and consistency#

SQLite gives transactional writes: a set of statements either all apply or none do. That matters for any operation that changes more than one row — moving an item between lists, for example.

DataStore gives atomic whole-value writes, which is enough for settings and not enough for multi-entity changes. A transaction that spans two DataStore keys is not a transaction at all.

Cache invalidation and stale data#

A cache directory has no invalidation contract. The system can delete it at any time, and the app cannot tell when.

The supported approach is to store the identity of the cached content alongside it — an ETag, a timestamp, a version — so a missing cache entry is a cache miss rather than a data loss. An app that treats cache as durable will lose data the first time the device runs low on storage.

Backup and secrets#

Backup includes app data by default, which is why an app that stores a token in SharedPreferences must exclude it — otherwise the token is copied to the user's cloud backup and restored on a new device.

The exclusion lives in the backup rules, not in the storage choice. Moving a secret to a different directory does not remove it from backup unless the rules say so.

The device-transfer distinction#

dataExtractionRules has separate sections for cloud backup and device-to-device transfer. A setting that should survive a phone upgrade may not be appropriate for a cloud restore, and vice versa.

The default is to include everything, which is why the rules are worth writing explicitly for any app with sensitive or device-specific state.

Migration between the two#

Moving from SharedPreferences to DataStore, or from raw SQLite to Room, is a one-time migration that must be idempotent. Read the old store, write the new one, mark the migration complete, and keep the old data until the new store is verified.

The pattern is the same as a database migration: never delete the source until the destination is known good.

Schema changes are a contract with the future#

A database schema is not just a description of today's data; it is a promise to every future version of the app that the data can still be read.

That is why a migration is a code path that must be tested, not a one-line ALTER TABLE written in a hurry. The Room schema export writes the schema of each version to a JSON file, which is what allows the test to open a database at version N and migrate it to N+1 in isolation.

When destructive migration is acceptable#

Dropping and recreating is acceptable when the data is a cache — re-fetchable, non-authoritative, and cheap to rebuild. It is not acceptable for user-generated content.

The distinction should be explicit in the code, not implicit in the absence of a migration, because a destructive migration that ships by accident is indistinguishable from one that was intended.

Cache sizing#

A cache with no size limit grows until the device runs out of storage, at which point the system deletes from the cache — possibly while the app is using it.

A well-behaved app caps its cache, evicts the oldest entries first, and treats a missing cache entry as a normal condition rather than an error.

The export path#

Data the user can take with them is different from data the app manages. An export is a SAF document, written through a grant the user gave, and it is the only way to get data out of the sandbox without a share intent.

That path is worth designing early, because it is also the path that a device-to-device transfer may not cover.

How it evolved#

Room and DataStore are AndroidX libraries rather than platform APIs, which means an app on an older release can still use them — the library ships with the app.²³

Under the hood#

• ‹›SharedPreferencesImpl.java↗the in-memory map and the async disk write
• ‹›SQLiteDatabase.java↗the platform SQLite wrapper
• ‹›↗the backup transport and the agent
• ‹›Context.java↗filesDir, cacheDir and the storage constants

Curiosity corner#

• SharedPreferences loads the whole file on first access. A large preferences file costs memory in every process that reads it.
• SQLite is in the platform, but Room is not. A Room bug fix ships through the app, not through an OS update.
• Device-to-device transfer is not the same as cloud backup. The rules can differ, which is why dataExtractionRules has separate sections.

Try it#

$ adb shell run-as com.example.app ls -la

drwx------  files
drwx------  cache
drwx------  databases
drwx------  shared_prefs

$ adb shell run-as com.example.app sqlite3 databases/app.db ".schema"

note  | CREATE TABLE note (id INTEGER PRIMARY KEY, title TEXT, body TEXT)

$ adb shell dumpsys backup | grep -A3 "com.example.app"

Backup enabled: true
Backup agent: com.example.app.BackupAgent

Check yourself#

Sources#