Tuesday, 3 December 2013

Android Partitions and Kernel Explained

The phone's internal memory (not the SD card) is solid-state (flash) memory, AKA NAND. It can be partitioned much like a normal hard drive can be partitioned. The bootloader exists in its own partition. Recovery is another partition; radio, system, cache, etc are all partitions.

Here are the standard partitions on an Android phone:

  1. /misc - not sure what this is for. 
  2. /boot - bootloader, kernel 
  3. /recovery - holds the recovery program (either clockworkmod or RA recovery for a rooted Evo) 
  4. /system - operating system goes here: Android, Sense, boot animation, Sprint crapware, busybox, etc 
  5. /cache - cached data from OS usage 
  6. /data - user applications, data, settings, etc.

The below partitions are not android-specific. They are tied to the hardware of the phone, but the kernel may have code allowing Android to interact with said hardware.

  1. /radio - the phone's radio firmware, controls cellular, data, GPS, bluetooth. 
  2. /wimax - firmware for Sprint's flavor of 4G, WiMax.

During the rooting process, a critical piece of the process is disabling a security system built into the bootloader that protects these partitions from accidental (or intentional) modification. This is what's referred to as "unlocking NAND." The security system can be set to active or inactive. S-ON means the security is in place (NAND locked). S-OFF means the security is off (NAND unlocked). When S-OFF, you have the ability to modify all partitions. With S-ON, you only have write access to /cache and /data. Everything else is read-only.

When you flash a custom ROM, that ROM typically includes a kernel and an OS. That means the /boot and /system partitions will be modified at a minimum. Some ROMs require a clean install, so a format of the /data and /cache partitions is sometimes built into the .zip that you flash. This is essentially doing a factory reset. See next paragraph.

When you do a factory reset (AKA: wipe, hard reset, factory wipe, etc.), you are erasing the /data and /cache partitions. Note that a factory reset does NOT put your phone back to its factory state from an OS standpoint. If you've upgraded to froyo, you will stay on froyo, because the OS lives in /system, and that is not touched during a factory reset. So "factory data reset," as it says under Settings > SD & phone storage, causes confusion. It's not a factory reset. It's a factory DATA reset. Now you know the distinction.

The SD card can also be partitioned to include a section dedicated to storing user apps. To create the partition, your SD card needs to be formatted. Typically a user will copy all the contents in the SD card to a PC hard drive, wipe the card and partition it, and then copy everything back.

Onto kernels....

A kernel is a layer of code that allows the OS and applications to interface with your phone's hardware. The degree in which you can access your phone's hardware features depends on the quality of code in the kernel. The homebrew (rooting) community for HTC has made several kernel code improvements that give us additional features from our hardware that the stock kernel does not. When you flash a custom ROM, you automatically get a kernel. But you can also flash a standalone kernel ROM on top of the existing one, effectively overwriting it. These days, the difference in custom kernels is less about new features and more about alternate configurations. Choosing a custom kernel is basically choosing one that works best with your ROM.

Kernel developers are typically responsible for a small, specific feature in the kernel. For example, netarchy's contribution to custom kernels mainly revolve around removing the framerate restriction that was present in the stock HTC kernel. However, because of the open-source philosophy of all the devs, each kernel "distribution" contains the work of several devs. They openly share new code that they deem will benefit everyone. Basically the best features stay in, and the distributor of the kernel will give credit where credit's due. This means that for any custom kernel you try, you might be using netarchy's FPS fix. It's all a team effort. A new improvement will make it to most if not all of the various kernel distributions in a short amount of time.

For the Evo, the hot features of custom kernels are:

  1. a fix to remove the 30fps cap imposed by HTC and improve touch tracking sensitivity (HTC finally fixed this in their stock kernel late last year)
  2. disabling perflock to enable CPU throttling. Great for increasing the performance of your phone and/or improving battery life
  3. iptables firewall to enable wifi-tethering via the wifi-tether app
  4. 3 or 5 point multitouch support (not too many practical applications)
  5. HAVS, a control system that dynamically adjusts the voltage based on CPU load. This has proven to be a battery saver, but it can actually have the opposite effect when multiple control systems are operating (like setCPU).
  6. BFS kernel task scheduler as an alternative to the standard CFS
  7. SBC (the ability to charge your battery beyond the default safe limit). The concept is similar to overclocking a processor: you're overriding the safety limits established to achieve additional performance. The benefit here is that you may gain more use of your battery per charge. The drawback is that you can damage the battery and significantly reduce its longevity. Some kernels claim they are using a safe technique to prevent battery damage. Just be aware of the potential risks.

Note about 4G and HDMI:
AOSP ROMs (ROMs based on pure Android without 3rd party manufacturer's code) are currently lacking robust 4G and HDMI support. This is because 4G and HDMI are hardware-specific features, not Android-specific, so there is no existing kernel code in the Android Open Source Project (AOSP) that is specific to 4G or HDMI support.

HTC is the manufacturer of the phone, so it has intimate knowledge of the hardware it uses in the phone; therefore the kernel that HTC has written does support HDMI and 4G. The problem is that the kernel is also tied in with their Sense UI that it's not possible to simply extract the portions of the code governing 4G and HDMI. Therefore, Sense-based ROMs support 4G because they simply borrow the existing HTC code. Meanwhile AOSP ROMs have to write their own code for 4G and HDMI in order to support it. Their ability to do this depends on how much info they can obtain about the hardware components in the phone and any drivers they may be able to find. Otherwise, it's really a task of reverse-engineering the hardware, which can be a thankless, time consuming task.

Note :  This is not my original writing. This is picked up from android forum.  I just though the knowledge would be more readable for beginners here than on the forums.

Android Partitions Basic

I. The Basics

Every operating system has a way of viewing and manipulating the filesystem(s) available to it. In the case of Windows, you have designated drive letters which represent the physical drives or partitions on a drive. Under each drive, you have folders and files. Those folders and files are data on each one of those specific drives. It's all very concise and obvious as to what data exists on what drive. There are things you can do complicate this, but this is not a discussion about Windows.

Android does not work anything like Windows, however, as it is a Linux based system. Linux/UNIX type systems have one top level file structure. The top level is called root, and is designated by /. There are no drive letters, and the files and folders under / are not necessarily all stored in the same physical location. Let me just take a very brief detour here. The top level file system is called root. When we say root in this case, we are not talking about the root user or root level permissions. When you root the phone, we're talking about have permissions to do things as the root user and make changes we couldn't otherwise do. The word root is used for different things. When you hear the word root used with filesystems, we mean the top level.

From this point on, the root of the file system will just be referred to as /. There really isn't much under / by itself. It's generally a small partition. In order to use other partitions, Android must make these partitions available under /. There is a special directory called /dev. Under this directory, you will find logical representations of partitions and resources available to the system. These, on their own, are just device files. These files exist solely so that the system knows they exist. You cannot try to view anything in them. So, in order to use partitions found in /dev, the system must mount them under /. This means the system must take a physical or logical media where you have some data stored, and make it available.

II. Mounting

When the system (or the user) initiates this process of mounting, a directory (folder) must exist somewhere under / for the partition to be mounted to. It can be directly under / or in a nested directory under /. We call this a mountpoint. What it means is that an empty directory will be designated to show the contents (data) of a partition. So, we say that internal storage is partitioned with various partitions such as /boot, /system, /data, /cache, etc. What this really means is that these partitions are all a part of internal storage, but have been logically separated so that the system sees them as separate as different physical media (i.e. partitions), but are then all mounted (made accessible) in one logical location. If you browse files on /system, then browse file on /data, you're under the root filesystem in both cases, but are looking in two different actual locations.

This type of thing is certainly not limited to partitions on internal storage. Any time you have a different physical media, it must be mounted under / just like any other partition. For example, you have a directory under / called /sdcard. That is a symlink (shortcut) to /mnt/sdcard, which is a mountpoint for the sdcard. More information on symlinks and how they work later. For now, just understand that it's basically a shortcut. Anyway, its partition can be found in /dev, as well, albeit in a different location. It is mounted under /mnt, accessible under /, all the same. The reason it's mounted under /mnt rather than / is because Linux based systems usually use /mnt for external device mounts. It doesn't have to, though. It could be mounted directly under / as /sdcard. It would still work. In any case, think of viewing the sdcard this way in the same manner that you would connect a usb flash drive to a Windows computer. When you do that, Windows sees the physical drive, then creates a letter drive for you to view it's contents. The principle is the same in that Android sees the physical media, but it mounts it in the same logical location (/) as all the other mountpoints. From a user's perspective, on Android, all the mountpoint directories look like they may as well be on the same media, even though they are not. On a Windows computer, when you are done with a flash drive, you have to tell it to safely eject the hardware. On Android (Linux), you have to do something similar, called unmount. If you do that to the sdcard, you will no longer be able to view /sdcard. Though you could create a directory under / called sdcard, it wouldn't have anything in it, and would be just a directory on internal storage until you remount the sdcard. Typically, the system mounts and unmounts partitions automatically when necessary.

There are two very important conclusions made from the above:

1) The system mounts and unmounts partitions at boot and shutdown. If you pull the battery out while the system is running or use a poorly written app to reboot the phone, partitions are still mounted, and if the system is writing to them, you could easily corrupt something. Granted, sometimes this is necessary if the phone becomes unresponsive. It's more likely to be a problem if repeatedly done.

2) Ever wonder why you cannot access the sdcard while the phone is connected to a computer as a disk drive? It's because the computer is mounting the sdcard partition so that you can see it there. What does that mean? It means Windows, Linux, etc. (whatever OS you have on your computer) has direct access to that physical media. No two operating systems can have direct control over physical media at the same time. It would result in massive data corruption. You may be wondering how it's possible to share drives or partitions in the networking world. You can do so because the Computer that has the physical media is still the only host that can physically read and write to the media. Sharing of the data is done at a much higher level and is controlled by the operating system.

III. Mount Options

Linux based systems have a file called fstab. That file is a mapping of physical partitions and their mountpoints, along with options it needs to know when mounting said partitions. It uses this file to mount partitions at boot time. So, the fact that you don't have to mount /system, /boot, /data, etc. yourself is because the system does it for you. It uses options it's told to use, though. There are various options that can be specified, but it is out of the scope of this explanation. What is important, though, is the designation between read/write (rw) and read-only (ro). This option is specified at boot time for automatically mounted partitions.

The partition mounted as /data is mounted as rw. It has to be, otherwise the system would be all but unusable. You wouldn't be able to install apps, change settings, etc. Do not confuse this with file permissions. That is a different discussion for a different time, but at least understand this - file and directories have certain permissions that allow the file owner, the group the file owner belongs to, and everyone else specific access to said file. Those permissions can be either the ability to read the file, write (modify) to the file, or execute it. The same goes for directories (though executing a directory doesn't make sense). The point is that a partition must be mounted as rw in order for write permissions to work. If you have permission to write to a file on a partition, but it is mounted as ro, it will not work.

Note : This is not my original writing. This is picked up from android forum.  I just though the knowledge would be more readable for beginners here than on the forums.
t> UA-39527780-1 back to top