How Android works

Media, audio & camera

Decoding, playing, capturing, and the vulnerability that changed Android.

5 pages55 min5 interactive7200 words

01The media frameworkPlaying a video is four problems in a trench coat: a container to parse, a codec to decode, a surface to draw into, and a clock to stay on. Android's media framework is the history of separating them.intermediate11 mininteractive
02AudioAndroid's audio server mixes every app's sound into one stream and hands it to hardware. The buffer between them is why Android audio was famously laggy, and why AAudio exists.intermediate10 mininteractive
03CameraCamera2 is hard for a reason: it exposes the actual hardware pipeline. CameraX exists to wrap the common cases without hiding that pipeline completely.intermediate12 mininteractive
04Media sessions and playbackA media session is how an app tells the system what it is playing, so the lock screen, the notification shade, the car and the headphones can all control it without knowing anything about the app.intermediate12 mininteractive
05DRM and codecsWhy some phones play HD and others do not: DRM security levels and codec capabilities are hardware claims, not software features, and the platform can only expose what the silicon supports.nerd10 mininteractive

Test yourself

Test yourself: Media, audio & camera

1 / 8

Which clock does a video player follow when synchronising audio and video?