[DEV][GUIDE][REF] Customize Internal Partition Layout for MTD Devices

I am trying to gather stock partition information for all Xperia 2011 HDPI devices to port a ROM (semi-official MIUI) and Kernel to. So far I only have zeus and phoenix ported. Please read the guide and help out if you can so I/we (Wedgess and I) can deliver an awesome kernel with massive internal data. Thanks!

See Post #2 for current known stock partition parameters. Cheers.

Introduction

This guide / reference aims to be a complete article on customizing, resizing and/or re-ordering the internal partition layout on most (any?) Android MTD-based device. I have seen many questions around the various forums on how to get more internal data so I thought I'd share my developments. Hopefully this will become a valuable resource for kernel builders/hackers.

The guide is especially valuable (and, in the case of my kernel builds, originally designed for) the Xperia 2011 line, but from what I know it could be applied to nearly any Android device where kernel source/flashing is possible.

I would like to gather stock partition information for various devices and place them into Post #2. If you can at least give me your Partition Info from ATAG (see "Gathering information" section), I can create a reference for all kernel developers. Thank you!

Requirements

• Android SDK
• Ability to build a kernel (this guide does not teach that)
• Ability to flash a kernel (unlocked bootloader, etc)
• Bootloader which exposes Partition info via ATAG on startup (see "Gathering information" section)
• Device which uses MTD Partition Map (Don't know how to check this, I think most Android devices do anyway)

Warning

As far as I know, following this guide and using precise calculation that you double and triple check before flashing will not result in a hard brick - however I can not guarantee it. Some devices may have "obscure" partition maps or some "protected" sectors for one reason or another, and modifying these *may* result in either a hard-brick (unlikely) or a permanent inability to restore your device to 100% stock (very likely) for warranty and/or DRM purposes. You have been warned. I/we/anyone may not be held accountable for any of these events should they occur, for you are doing this at your own risk and with your own responsibility.


Gathering information

1. The first thing you'll want to do for the sake of accuracy is to flash to a 100% stock firmware. In the case of Xperia 2011 devices, flash the latest stock FTF for your device.
2. With the stock firmware now installed, the second thing you will need to do is to flash a custom kernel that is rooted and has busybox installed. In most cases, a CM7/9/10 kernel will do nicely.
3. Power-off your device. Execute the following command from shell/console, and then press enter:
   
   Code:

   ---

   adb wait-for-device && adb shell dmesg

   ---

4. After pressing enter, the console will wait at the prompt as intended. Now, power on your device and immediately plug in the USB cable. I assume the ADB drivers are already installed.
5. Shortly (5-15 seconds) you should see a mass output from the kernel followed by a return to your shell prompt. If you don't, either your kernel or bootloader does not support it. Try a different kernel. If you still don't, then sorry but I think we can't do it for your device.
6. Scroll right to the top of the dmesg output, you should see something similar to this:
   
   Code:

   ---

   <6>[    0.000000] Initializing cgroup subsys cpu                                                                                                         
<5>[    0.000000] Linux version 2.6.32.9-KRsH (CosmicDan@Ubuntu, Linaro 4.7) (gcc version Linaro 4.6.2 20111004) #8 PREEMPT Thu Oct 25 15:57:27 EST 2012 
<4>[    0.000000] CPU: ARMv7 Processor [511f00f2] revision 2 (ARMv7), cr=10c53c7d                                                                         
<4>[    0.000000] CPU: VIPT nonaliasing data cache, VIVT ASID tagged instruction cache                                                                   
<4>[    0.000000] Machine: zeus                                                                                                                           
<6>[    0.000000] Partition (from atag) system -- Offset:2e4 Size:9c4                                                                                     
<6>[    0.000000] Partition (from atag) userdata -- Offset:ca8 Size:be0                                                                                   
<6>[    0.000000] Partition (from atag) cache -- Offset:1cb4 Size:32c                                                                                     
<6>[    0.000000] Partition (from atag) appslog -- Offset:1888 Size:42c                                                                                   
<4>[    0.000000] Memory policy: ECC disabled, Data cache writeback

   ---

   ...see those "Partition (from atag)" lines? That's what we need! Copy this information down and move on to the next section.

Additional verification and hidden partitions (optional)


As far as I know, this is only possible with Xperia 2011 devices. If you know of a method for other devices, please let me know.


We can additionally verify the ATAG information and map extra "hidden" partitions such as boot (kernel) by examining the SIN files inside an FTF. I will assume that you know how to use Flashtool already as I won't go into much detail here.


First, we need to enable the development features of Flashtool. In the program folder, open "config.properties" and edit/add the line like so:
Next, extract your stock FTF bundle with any ol' ZIP extractor, load Flashtool, and select "Tools" > "SIN Editor", and open a particular SIN file that you want to verify/unhide. In this example, we will open system.sin. The "Partition Info" field is what we want. Behold:
The second last 8-bytes are the offset and the last 8-bytes are the size. By "byte-reversed" I mean that you read each byte from end to beginning, but not swap the bytes themselves. Thus the size above, reading "C4090000" is actually "000009C4". And as you can see, this matches 100% to our ATAG of 9c4 for system size. Correct sir! Additionally, the offset of "E4020000" > reversed to "000002E4" also matches.


So now, we can open "kernel.sin" and do the same, because we also want to partition "boot" (why not?). In this device (Xperia Play/Xperia Neo L), kernel.sin gives us 03000000220000808002000064000000 which means that the size is 64 and the offset is 280.



Calculations


From the partition info via ATAG, we can now build "stock" mtdparts information to apply to our kernel. Using the example(s) above, we can now build this information:


Note the syntax of size@offset. Next, we must convert the hex values to decimal, then multiply by 128 (I do not know if 128 is the same multiplier for all devices, please double check and let me know). This will give us the exact sizes and offsets in kilobytes.


Alright, so that is a 100% stock partition map for this device - except we also have the boot (kernel) mapped now too. Here is a (crappy) visual representation of it:


You may have noticed that the order we (and ATAG) lists the partitions in does not match the actual order of the partitions. It is quite important to retain the order of the partitions as specified in ATAG, because that's the order they will be mapped in. I.e. system will remain mtdblock0 and cache will remain mtdblock2. Any extra partitions should always go after these defaults.





Formatting for kernel


To specify the mtdparts parameter for the kernel to use is trivial. Doing this will over-ride the ATAG map (from bootloader) and everything in the system - including Recovery - will see your map from the kernel. Search your default config file in your kernel for the value "CONFIG_CMDLINE" and you should find a value like this :
Using the information we have above about our partition map, we add a new parameter here with "mtdparts=msm_nand:". The syntax is as follows:
Remember that we converted our size and offsets to kilobytes (for better readibility) so we need to specify size unit of k. The new parameter, from our above examples, becomes this:
Don't forget to retain the order! And so, our full line looks like this:
NOTE: Depending on your kernel, you may also need to enable the following line in your config:
And we're all done. If you build your kernel now, you will be able to mount (or at least dd from) the appslog and boot partitions.




Resizing partitions


This is somewhat trivial, the most difficult part is probably over - but this step can be tedious, albeit not very complicated. Anybody with an above average IQ would have already figured this out - we simply change the size of one partition and adjust the offsets of it's following partitions to accommodate for the change. Here is one good example that I use for the MIUI ROM for the Xperia Play and Neo L, compared to the stock examples shown above:
...and a visual representation of this new map:
Hopefully that's enough to make sense. Remember to verify your modified partitions. This can easily be done by adding the size+offset of a partition, giving the offset of the next partition. E.g. in this mod, userdata ends at 1036288 (654848+381440) which matches the offset for the next partition - cache.




Troubleshooting/Recovering from modified partitions


In some cases, your new kernel may not boot. A common issue is that the kernel logo will show, and the device will shortly reboot (kernel bootloop). This can be solved by formatting your partitions with fastboot after flashing the new kernel, usually system and userdata are all that is needed:

If you wish to return to a stock partition layout, sometimes flashing a non-modified kernel is not enough. You may get stuck on kernel logo even after formatting system and userdata. In this case, flashing a stock Firmware and setting your phone back to scratch should result in a 100% original device. But if your phone is still bricked, sorry but it's not my fault. You probably did something wrong.





#####



OK, that's the guide done for now. Any questions or suggestions on the guide, please let me know! Also, refer to post #2 for some stock partition map reference.