This blog post is a story with a very happy ending. The story explains how a small number of highly-talented, highly-committed Linux engineers were able to reintegrate the worst fork that the Linux world has yet seen. Once they made up their minds to do it, they did it calmly, quietly, and effectively. Their accomplishment is legion, but as yet known to few people in the Android world. These guys exemplify the best aspects of the open source community. I'm getting ahead of myself, here. Let's go back to basics.
The Android framework runs on top of a Linux kernel that has some Google-only changes. These features aren't part of, or are done a different way to, mainstream Linux. Examples are:
- wake locks to keep a screen powered on ("lock it into the awake state")
- the Binder component for IPC
- the pmem allocator for large amounts of contiguous memory (used for e.g. video capture buffers)
- the "low memory app killer"
- the RAM console, which is actually a debugging mod. It's a protocol for saving log messages to an area of memory that is not cleared by a warm reboot. A crashed system can reboot without losing vital diagnostic messages.
By "Google-only changes", I mean that the Google code was not being merged back into the mainstream Linux repositories. The usual merging back wasn't done because people with good intentions and differing priorities disagreed about technical aspects of the changes. (Such differences occur frequently in open source kernel programming).
Some changes were fairly innocuous. The "low memory app killer" sounds like something no desktop Linux user (with disk-backed virtual memory) would ever want. But it was implemented in a technically adept way - the low memory killer uses a standard Linux feature to register a "low memory listener" (in Android terms). The kernel will call that event handler when it needs more memory. As a result, Android's low memory killer can easily be ignored in systems that don't wish to use it. You can review the C code for the low memory killer here. It's in the staging area now, ready to push into the mainline.
"Android and the Linux Kernel Mainline" panel at the 2012 Android Builders Summit.
Left to right: Zach Pfeffer, Tim Bird, Arnd Bergmann, and Greg Kroah-Hartman
Other Google-only changes were more substantial. Binder is a piece of Android that originated with engineers from BeOS. The problem with bringing Binder into the Linux kernel is that there are several pre-existing system components that do a similar job of managing inter-process communication. Alternatives such as kernel-dbus and Cross Memory Attach each have unique aspects. Binder's unique aspect is that it doesn't copy any areas of memory between the two processes when doing an IPC, and is thus fast. You can review the kernel code for Binder here (for interest only - it won't help you with any Android development).
Linus Torvalds (the creator of Linux) played a part in resolving the engineers' differences over the Google kernel changes. At the 2011 Kernel Summit, the topic came up that many Android changes were blocked from merging. Linus responded that code that is actually in use is worth something. It's certainly true that Android code is in use. Andy Rubin (Google's Android VP) noted at this year's Mobile World Congress that there are more than 300 million Android devices in use. If you think of Android as a Linux distro (and I think we should), it is almost certainly the distro with the largest number of users. I write "almost certainly" because most distros don't have a good way to track how many users they have. As a complete SWAG on my part, the next most numerous distro, Ubuntu, probably has one order of magnitude fewer users than Android. (Update: Canonical tracks user numbers by counting the unique IP addresses that hit its servers for security updates. Two years ago, they said they had 12 million users. Last year, they said they wanted 200 million users by 2015 - so my SWAG for today's numbers was right on target).
In any event, Linus's penchant for high volume projects inspired Tim Bird, who in December 2011 announced the Android Mainlining Project to bring most of the Android changes back into the Linux kernel. Much of the integration work was done by Greg Kroah-Hartman. And now with the March 2012 release of Linux kernel version 3.3, the first part of that work is successfully completed.
What does it mean for the Android changes to be merged with the mainstream Linux kernel?
- for App developers, directly, nothing (sorry).
- for the Android kernel team at Google, it reduces the patch maintenance burden. They no longer have to find and port bug fixes from the mainstream. They can adopt newer versions of the kernel more easily.
- for hardware manufacturers, again a saving of unnecessary work. They can download a kernel from one of the repositories, and expect it to boot on their Android device with minimal changes. Developers and system integrators will still need to go to Google for specific features that are not mainlined, when they are putting together a new product.
- some hardware manufacturers will get a reduction in kernel management complexity internally, by using a single
(shared or overlapping) kernel tree for both their Android and non-Android Linux-based products. - for systems integrators, again unnecessary work is avoided
- for the Linux ecosystem, it is now much easier for people to experiment with Android on mainline kernels, especially
on new hardware unsupported by Google. - for the Android ecosystem, it is now much easier for people to experiment with different Linux versions.
The bottom line is that this work saves money and speeds up innovation. The most significant fork of Linux is well on the way to reintegration. Next mission: replace Android's version of libc (called "bionic") with something that implements the POSIX API.
References
1. Linux kernel version 3.3 content
2. The patch review discussion at the 2011 Linux Kernel Summit
3. A very informative Linux Weekly News article by Jonathan Corbet on mainstreaming Android.
Acknowledgement: I'm grateful to Tim Bird, who generously helped me improve this blog post. I learn a lot from the writings of Jonathan Corbet on the topic of Linux.
