[GUIDE] Learning about the Android Build Process

Introduction

The Android open-source project (AOSP) is quite complex and it can be hard to find a good way to get more familiar with it. I’m going to try a practical approach, by describing the relevant parts of the build process.

Note - I recommend reading my this guide before you through this one just in case you are actually gonna read and learn. :)

This guide explains the whole android build process right from envsetup.sh to the working and understanding of makefiles to Package complete! :D

The Beginning(envsetup.sh)

It all gets started with the
command. What it basically does is that it adds a number of commands to your environment which you later use for building Android for your device.

The commands are listed below -
It also scans for the 'vendorsetup.sh' files in your source and display them in this form -
The file is responsible for -

1. All the make device calling, make etc. commands are possible only after you execute the envsetup.sh file. :)
2. It also scans and does some magic stuff with the build/core directory.
3. It also displays choices of variant for the selected device, like 'userdebug', 'user', 'eng'.
4. It is also responsible for performing the most important step, it sets up the android build paths and and sets up the toolchains for the build process.
5. It also calls java and sets it parameters for the compilation.
6. It displays the default device configurations that are -
   
   Code:

   ---

   add_lunch_combo aosp_arm-eng
add_lunch_combo aosp_x86-eng
add_lunch_combo aosp_mips-eng
add_lunch_combo vbox_x86-eng

   ---

7. It calls the 'roomservice.py' file in build/tools folder which is responsible for finding and downloading the device sources from CyanogenMod's github if they don't exist.
8. The file also contains the CWM and TWRP commands to sideload the product files into the device.
9. It also gets bug reports from the device to the machine for the developers to work upon. :)

The lunch/brunch Command

Next thing is to select target using the lunch menu, which was added to your bash shell environment after sourcing envsetup.sh. After making your selection, the chosen product and variant is verified and environment variables are set, including:

• export TARGET_PRODUCT=$product – The chosen product
• export TARGET_BUILD_VARIANT=$variant – The chosen variant
• export TARGET_BUILD_TYPE=release – Only release type is available. Use choosecombo if you want to select type.
• export ANDROID_BUILD_TOP=$(gettop) – The build root directory.
• export ANDROID_TOOLCHAIN=... – The toolchain directory for the prebuilt cross-compiler matching the target architecture
• export PATH=... – Among other stuff, the prebuilt toolchain is added to PATH.
• export ANDROID_PRODUCT_OUT=... – Absolute path to the target product out directory
• export ANDROID_HOST_OUT=... – Absolute path to the host out directory

The usage is pretty simply, you can just type 'lunch' and you'll be prompted with the list of devices that you can build for with the sources present or you can just type 'lunch cm_<device_name>-<variant_type>'.
The lunch command actually fixes the target for the build process and is responsible for the paths getting locked and the device configurationto be used and applied to the source. :D
It actually gets to use only after there is a vendorsetup.sh file in your device tree which is responsible for adding the lunch combo(Which shows up in the list of devices when you simply run lunch) of your device.

The MAKE Command

The purpose of the make utility is to determine automatically which pieces of a large program need to be recompiled, and issue the commands to recompile them. The manual describes the GNU implementation of make, which was written by Richard Stallman and Roland McGrath. The following examples show C programs, since they are most common, but you can use make with any programming language whose compiler can be run with a shell command. In fact, make is not limited to programs. You can use it to describe any task where some files must be updated automatically from others whenever the others change.

To prepare to use make, you must write a file called the makefile that describes the relationships among files in your program, and the states the commands for updating each file. In a program, typically the executable file is updated from object files, which are in turn made by compiling source files.

Once a suitable makefile exists, each time you change some source files, this simple shell command:

suffices to perform all necessary recompilations. The make program uses the makefile data base and the last-modification times of the files to decide which of the files need to be updated. For each of those files, it issues the commands recorded in the data base.

make executes commands in the makefile to update one or more target names, where name is typically a program. If no -f option is present, make will look for the makefiles GNUmakefile, makefile, and Makefile, in that order.

Normally you should call your makefile either makefile or Makefile. (I recommend Makefile because it appears prominently near the beginning of a directory listing, right near other important files such as README.) The first name checked, GNUmakefile, is not recommended for most makefiles. You should use this name if you have a makefile that is specific to GNU make, and will not be understood by other versions of make. If makefile is `-', the standard input is read.

make updates a target if it depends on prerequisite files that have been modified since the target was last modified, or if the target does not exist.

Makefile Execution

The make command executes the commands in the makefile line by line. As make executes each command, it writes the command to standard output (unless otherwise directed, for example, using the -s flag). A makefile must have a Tab in front of the commands on each line.

When a command is executed through the make command, it uses make's execution environment. This includes any macros from the command line to the make command and any environment variables specified in the MAKEFLAGS variable. The make command's environment variables overwrite any variables of the same name in the existing environment.

Note:
Environment: The make command uses the MAKEFLAGS environment variable, if it exists.

Target Rules

Target rules have the following format:

target[target...] : [prerequisite...] [;command]
<Tab>command
Multiple targets and prerequisites are separated by spaces. Any text that follows the ; (semicolon) and all of the subsequent lines that begin with a Tab character are considered commands to be used to update the target. A new target entry is started when a new line does not begin with a Tab or # character.

You can also look up at the Makefile Tutorial here http://mrbook.org/tutorials/make/ !

That was the most important part of the build process, where make files are resposible for compiling almost each of the files in the source and putting them together for the useful apps/binaries/libraries etc. :D

Build Tricks

Used from wiki.

Seeing the actual commands used to build the software
Use the "showcommands" target on your 'make' line:

$ make -j4 showcommands
This can be used in conjunction with another make target, to see the commands for that build. That is, 'showcommands' is not a target itself, but just a modifier for the specified build.

In the example above, the -j4 is unrelated to the showcommands option, and is used to execute 4 make sessions that run in parallel.

Make targets
Here is a list of different make targets you can use to build different parts of the system:

• make sdk - build the tools that are part of an SDK (adb, fastboot, etc.)
• make snod - build the system image from the current software binaries
• make services
• make runtime
• make droid - make droid is the normal build.
• make all - make everything, whether it is included in the product definition or not
• make clean - remove all built files (prepare for a new build). Same as rm -rf out/<configuration>/
• make modules - shows a list of submodules that can be built (List of all LOCAL_MODULE definitions)
• make <local_module> - make a specific module (note that this is not the same as directory name. It is the LOCAL_MODULE definition in the Android.mk file)
• make clean-<local_module> - clean a specific module

Helper macros and functions
There are some helper macros and functions that are installed when you source envsetup.sh. They are documented at the top of envesetup.sh, but here is information about a few of them:

croot - change directory to the top of the tree

Speeding up the build

You can use the '-j' option with make, to start multiple threads of make execution concurrently.

In my experience, you should specify about 2 more threads than you have processors on your machine. If you have 2 processors, use 'make -j4', If they are hyperthreaded (meaning you have 4 virtual processors), try 'make -j6.

You can also specify to use the 'ccache' compiler cache, which will speed up things once you have built things a first time. To do this, specify 'export USE_CCACHE=1' at your shell command line. (Note that ccache is included in the prebuilt section of the repository, and does not have to be installed on your host separately.)

Building only an individual program or module
If you use build/envsetup.sh, you can use some of the defined functions to build only a part of the tree. Use the 'mm' or 'mmm' commands to do this.

The 'mm' command makes stuff in the current directory (and sub-directories, I believe). With the 'mmm' command, you specify a directory or list of directories, and it builds those.

To install your changes, do 'make snod' from the top of tree. 'make snod' builds a new system image from current binaries.

Setting module-specific build parameters
Some code in Android system can be customized in the way they are built (separate from the build variant and release vs. debug options). You can set variables that control individual build options, either by setting them in the environment or by passing them directly to 'make' (or the 'm...' functions which call 'make'.)

For example, the 'init' program can be built with support for bootchart logging by setting the INIT_BOOTCHART variable. (See Using Bootchart on Android for why you might want to do this.)

You can accomplish either with:

$ touch system/init/init.c
$ export INIT_BOOTCHART=true
$ make
or

$ touch system/init/init.c
$ m INIT_BOOTCHART=true

At last, after makefiles optimize all the processes and build the device specific parts including binaries and libs and apps necessary for it to get booted, the 'system' folder and the 'boot.img' folder are prepared in the out/target/product/device. The META-INF folder is prepared at instance and the system and boot.img are packed into a zip file(whose name is also processed by the makefiles ;) ) and md5 sum prepared. The flashable zip gets prepared only if you run the "brunch" command or "lunch + mka" command. :)

The Build Tricks aren't *for fun*. This stuff is always gonna help you in the long run! :D

Credits -
>Firstly, my parents. :)
>Google, for Android!
>All the people over xda who have written awesome guides and done great works(Because that's what inspires me)
> @galaxyfreak , @speed_bot, @Red Devil, @thewisenerd, @vishal_android freak and all others who helped me every time and answered every(sensible) question of mine.
>The CyanogenMod, AOKP and OmniROM team! :)
> All those whom I missed. :)

Have a Happy time learning! :)