How does audio work on Android?

Android devices support a wide range of audio capabilities that enable apps to play music, record audio, apply audio effects, route audio to different outputs, and more. Here’s a brief overview of some of the key audio features available on Android:

Android devices typically include built-in microphones and speakers, along with audio jacks for connecting headphones or external speakers. Many devices also include more advanced audio hardware like stereo speakers, hi-fi DACs (digital-to-analog converters), and powerful amps for driving high-impedance headphones.

The Android framework provides APIs for audio playback, recording, and management in the android.media package. Developers can use these APIs to play audio files, streams, and generate tones programmatically. The MediaRecorder APIs allow recording from microphones and other inputs.

Android supports standard audio formats like MP3, AAC, FLAC, PCM, OGG, and MIDI. Apps can decode these formats in software or leverage hardware decoding on supported devices. Audio routing options allow directing audio to built-in speakers, headphones, Bluetooth devices, and more.

In summary, Android provides a rich set of audio capabilities via its hardware support and software APIs, enabling developers to build immersive music, podcast, and audio apps. The rest of this article will explore Android’s audio stack in more depth.

Audio Hardware

Android devices typically include standard audio hardware components like speakers, microphones, headphone jacks, and internal audio chips/drivers. Here are some key hardware elements for Android audio:

Speakers – Most Android phones include built-in speakers for playing music, system sounds, ringtones, notifications, etc. Speakers may be mono or stereo. Quality varies across devices from basic tiny smartphone speakers to dual front-facing speakers in premium devices like the Google Pixel.

Microphones – Microphones allow recording audio and enable features like phone calls, voice commands, videos, and voice messages. Most Android phones have at least one built-in microphone, often located at the bottom or top of the screen bezel. Some devices like the Samsung Galaxy S series have multiple mics for noise cancellation.

Headphone Jacks – Many Android phones still include a 3.5mm headphone jack for connecting wired headphones, earbuds, aux cables, etc. However some manufactures like Google and OnePlus have dropped the headphone jack in recent models to save space. Alternatives include USB-C audio, Bluetooth, and adapters.

Audio Hardware Chips/Drivers – Every Android device has internal audio hardware like digital-to-analog converters (DACs), amplifiers, and audio drivers from companies like Qualcomm, MediaTek, Cirrus Logic, and Realtek. The drivers and silicon chips process and output digital audio signals to the device speakers and headphone jack.

According to the Android source overview, the audio hardware directly interacts with drivers and the Android HAL implementation to handle audio functionality. Common Android audio hardware provides the underlying components needed for audio playback, recording, and connectivity.

Audio Software and APIs

Android provides several APIs and frameworks for working with audio. Some key ones include:

The MediaPlayer class can play audio from files or network streams. It supports playback of common formats like MP3, AAC, OGG, FLAC, MIDI, WAV, and more. MediaPlayer handles decoding audio and outputting it to speakers or headphones. It provides controls for playback and volume. Source

The AudioManager deals with audio configuration and routing. It can do things like set speaker mode, adjust stream volumes, handle wired headset connections, and redirect audio output to Bluetooth devices. AudioManager has features for audio focus to coordinate playback from multiple apps.

The lower level AudioTrack API outputs raw PCM audio buffers. This gives apps more control for custom usage like audio synthesis, realtime audio processing, voice chat, etc. AudioTrack handles interacting with audio drivers and hardware. It has options to tune buffer size, sample rate, channel count, encoding and more. Source

Audio File Support

Android supports a wide variety of audio file formats and codecs out of the box. Some of the most commonly used audio file types that are natively supported include:

• MP3 – Android supports mono/stereo MP3 files from 8-320Kbps in both constant bitrate (CBR) and variable bitrate (VBR) formats (Supported media formats). MP3 is one of the most popular audio formats due to its efficient compression and widespread compatibility.
• FLAC – Free Lossless Audio Codec (FLAC) is an open source lossless audio format that compresses audio without any loss in quality. FLAC files are supported natively on Android.
• OGG – The Ogg container format with the Vorbis audio codec is supported. Ogg offers better quality than MP3 at similar bitrates.
• MIDI – MIDI files are lightweight and widely compatible. Both MIDI Type 0/1 files and DLS banks are supported.
• M4A/AAC – Android natively supports both M4A and AAC files in the MPEG-4 container. AAC often exceeds MP3 in audio quality.
• WAV – Uncompressed WAV audio files are supported, though they have large file sizes.
• AMR – Adaptive Multi-Rate codec often used in voice recordings.

Android can play audio from these popular file types out of the box, with no need to download additional apps or codecs. MediaPlayer and other audio APIs automatically determine and utilize the appropriate decoder for these file formats.

Audio Encoding and Decoding

Android devices rely on dedicated hardware and software components for encoding and decoding audio. At the hardware level, most Android devices contain audio encoder and decoder chips that handle common formats like MP3, AAC, and more. The codecs offload intensive encoding/decoding tasks from the main processor.

On the software side, Android’s MediaCodec API provides interfaces for encoding and decoding audio using platform codecs. Developers can use MediaCodec to access hardware accelerators and encode or decode audio streams efficiently¹. By passing raw PCM samples into a MediaCodec encoder instance, you can generate compressed audio in formats like AAC, MP3, FLAC, etc. Likewise, passing encoded streams into a decoder outputs raw PCM audio samples.

MediaCodec encoders accept uncompressed audio in various sample formats like 8/16/24/32-bit and float. Decoders output raw PCM samples that can be played back or further processed. Some advantages of using MediaCodec include reduced CPU load, lower latency, and better power efficiency versus software encoding/decoding.

Overall, Android’s hardware acceleration and APIs like MediaCodec optimize audio encoding and decoding. They allow efficient compression and decompression while minimizing battery drain on mobile devices.²³

Audio Routing

Audio routing refers to how audio is directed to different outputs on an Android device like the built-in speaker, headphones, Bluetooth devices, HDMI, USB audio, etc. The audio routing is handled by the Android audio framework and varies across different Android versions and device manufacturers.

On most Android devices, plugging in headphones will automatically route audio to the headphones instead of the built-in speaker. Bluetooth audio devices like speakers and headphones can be paired with the device and selected as audio outputs in the Bluetooth settings. Starting with Android 10, apps can programmatically request audio routing to specific outputs using the AudioPlaybackCaptureConfiguration API (https://xdaforums.com/t/android-audio-routing.3536668/).

However, the audio routing implementation can differ across manufacturers. For example, Samsung devices have their own proprietary audio routing APIs while the implementation on Pixel phones aligns more closely with stock Android behavior. There are also inconsistencies in how audio routing behaves when a phone call is in progress, as the cellular audio usually takes priority (https://stackoverflow.com/questions/66851768/how-does-being-in-a-call-affect-android-audio-routing).

In summary, Android audio routing allows directing audio to different outputs but its behavior depends on the Android version, device manufacturer customizations, and other factors like active phone calls. Apps can programmatically request specific routing from Android 10 onwards.

Audio Effects

Audio effects provide ways to modify and enhance audio playback on Android devices. Android includes built-in audio effects that developers can leverage through the Audio Effects API.

Some of the key built-in audio effects available include:

• Equalizer – Allows adjusting frequency levels to customize the sound.
• Bass boost – Emphasizes low frequencies to increase bass.
• Virtualizer – Expands the stereo image for a wider soundstage.

These effects are applied to the audio stream during playback using the AudioEffect class. Developers can enable effects like equalizer and bass boost to give users more control over the listening experience.

In addition to built-in effects, developers can also create custom audio effects by implementing the AudioEffect interface. Custom effects provide more flexibility but require more development work.

Audio Latency

Audio latency refers to the time it takes for an audio signal to be generated, processed, and output as sound from a device. On Android, latency has historically been higher compared to other mobile platforms like iOS, posing challenges for real-time audio applications like gaming, music production, and voice calls.

Several factors contribute to audio latency on Android devices:

• Hardware components like audio drivers, DACs, and SoCs
• Android audio software stack including buffers, effects, and audio routing
• The audio engine and thread priorities
• Interactions with other system processes and hardware

Android has undertaken efforts to reduce latency over the years. Strategies include:

• Low-latency audio APIs like AAudio introduced in Android 8 Oreo.
• Improvements to the audio stack and drivers through projects like Generic Audio HAL.
• Optimizations in Android 10 and 11 focused on faster buffer sizes and threading.
• Collaborations with chipset manufacturers to improve SoC capabilities.

While latency varies across Android devices, recent versions have achieved under 40ms latency on capable hardware. Further improvements to reach iOS levels of sub-10ms remain an ongoing priority.

Common Audio Use Cases

There are several common use cases for audio on Android devices:

Music and Video Playback
One of the most common uses of audio is music and video playback. Android provides audio APIs like MediaPlayer to play audio/video files, handle codecs and audio effects, manage audio focus between apps, and route audio to different outputs like headphones, speakers, or Bluetooth.

Voice Calls

Another major audio use case is voice calls over cellular networks or VoIP apps. The AudioManager handles routing call audio to the earpiece, speakerphone, wired headset, or Bluetooth headset. There is also noise suppression and echo cancellation to ensure call quality.

Recording
The android.media.MediaRecorder API allows capturing audio from the microphone for uses like voice memos, audio notes, or recording calls with consent. Audio recording has options like encoding, source selection, and effects.

Accessibility
Audio plays a key role in accessibility services for vision-impaired users. Text-to-speech converts text into audible speech using engines like Google Text-to-speech. There are also screen readers, like TalkBack, that can read screen content aloud. Audio cues aid navigation for accessible apps.

Conclusion

In summary, Android’s audio architecture has continued to evolve over the years to provide robust audio capabilities on mobile devices. The core components like the audio HAL, audio policy manager, audio flinger, and audio framework APIs allow apps to leverage advanced audio functionality. Android supports a wide range of audio codecs, effects, routing options, and low latency audio for pro audio applications.

Looking ahead, the future is bright for mobile audio as Android pushes the boundaries. Google is investing heavily in improving Android audio with initiatives like Project Acoustic. Newer versions of Android will likely bring enhanced audio quality, lower latency, better MIDI support, and more customizable audio routing. The proliferation of mobile devices running Android makes it a major audio platform that will shape the future of how we create, listen to, and interact with audio.