One of the common questions I get now and then on IRC (#OpenPandora on FreeNode) or email concerns how the OpenPandora booting and firmware is setup, what technologies are used in it’s setup and how we try and prevent things like accidental bricking and the like.
With this in mind I thought I would knock up a quick post detailing how these things are setup as a starting point to expand on.
I’ll start off with a technical overview of how the system boots and then detail how this will work for normal end users (and that includes ‘most’ developers).
There are 3 main sections to this post.
Only of interest to hacker types ;).
Normal users who want to play with alternative firmware, boot there own firmware and such look here.
Want to know a little about how is the stock firmware organised on the NAND? What setup is used? etc. – read this.
Booting the OpenPandora: The ‘low level’ process.
This OMAP3 features a multi-stage boot process that starts with a hardware boot ROM (burnt into the SoC and totally unchangeable) and ends with the booting of some appropriate code that has full control of the system.
Stage 0: The OMAP3 hardware boot ROM.
One of the good points of the OMAP3 SoC’s is that they feature a robust hardware boot ROM that can be configured to act in several consistent, pre-determined, ways.
The boot ROM is limited, all it can do is get up to 64k of code into internal SRAM and pass control to it. This is however enough to offer a pretty flexible (and ‘brick-proof’) setup at the lowest level.
We utilize this to accomplish the initial factory flashing of units and provide our ‘final backstop’ should a unit get corrupted. Should the NAND of a unit be totally trashed you can use this hardware boot ROM to boot a correctly configured SD card in slot 1 and restore the entire system to a working state.
We have configured the hardware boot ROM to try booting as follows:
Power on normally:
NAND –> USB Client –> UART (Serial) –> SD
Power on holding START button:
USB Client –> UART (Serial) –> SD –> NAND
Other OMAP3 boards may do this in a different order. It’s a characteristic of the board design.
This is called the ‘peripheral boot mode’ (and order) of the OpenPandora.
Please note that we are talking about the HARDWARE boot ROM not the software layer on top at this stage, this is an important distinction to make as most end users, and developers, will never need to have any dealings with the hardware boot ROM and it’s modes.
What the hardware boot ROM does for each type of ‘boot’ is summarised below.
- NAND Boot
The ROM looks for hashed startup code (normally a version of X-Loader hashed with ‘SignGP.c’) in the 1st sector of the NAND. If the sector is blank or corrupt the ROM will check the next 3 sectors before exiting. Once a good sector is found, the ROM transfers the contents to SRAM and transfers control to it.
- USB Client Boot
A vendor ID (Ti) and device ID (the model of the SoC) are sent out to the USB client port and should suitable software on the receiving end interpret and process this information the boot ROM will read data from the USB client port and transfer the data to SRAM. Control is passed to the start of SRAM if no errors are detected. The code transferred over USB is normally an unhashed version of X-Loader.
- UART (Serial) Boot
A special ID is written out to the serial port (on the OpenPandora EXT connection) and if it gets a response within a short period of time, the ROM will will read from the serial port and transfer the data to SRAM. Control is passed to the start of SRAM if no errors are detected. The code transferred over serial is normally an unhashed version of X-Loader.
- SD Card Boot
The ROM looks for an SD/MMC Card on the first MMC controller. If a card is found, the ROM then looks for the first FAT32 partition within the partition table and ensures that the ‘bootable’ flag is set on that partition.
If a suitable partition is found the root directory is scanned for a hashed file called ‘MLO’ (Normally a version of X-Loader hashed with ‘SignGP.c’) is read. Assuming all is well with the file, it is transferred into SRAM and transfers control to it.
Now we move on to the remaining stages of the boot process.
Stage 1: MLO (X-Loader)
Once the peripheral boot has taken place, by whatever means, you will be left with a copy of X-Loader running on the device. X-Loader is really just a massively cut down version of U-Boot designed to fit in the limited initial SRAM on the OMAP3.
Once it is running it will do some very specific checks, turn on a few extra components and jump to trying to load U-Boot for stage 2.
Stage 2: U-Boot
I won’t dwell on U-Boot here, it is a very well documented and well used bootloader used on lots of embedded platforms.
We have an OpenPandora specific U-Boot tree that we use to keep our platform specific patches in, most of our changes have long since gone into the mainline U-Boot trees.
U-Boot’s role is to setup most of the peripherals and load the kernel that will take control of the system. It also, in our case, provides a boot menu and a few additional tweaks to help developers.
Stage 3: Linux Kernel
This is the final stage of the boot process, control is handed from U-Boot to the Linux kernel and it is then up to Linux to get the system booted and running.
Booting the OpenPandora: End user booting and the boot menu.
So, having outlined the technical info on the OpenPandora boot system I am now going to mention that, all being well, most end users will never have any call to try and understand it.
As long as your unit has X-Loader and U-Boot correctly running from the NAND (as all units do out of the factory) you can take advantage of quick booting different OS’es from SD cards and tweaking the boot process without any need to create customised ‘fiddly’ bootable SD’s or compiling hashed copies of X-Loader and U-Boot.
In order to try and keep end users from having to get familiar with the quirks of the OMAP3 hardware boot ROM we (well Gražvydas Ignotas really) have implemented a number of features into the version of U-Boot that is loaded onto every unit.
One of the handier patches we carry in our U-Boot tree is the addition of a simple boot menu. This can be accessed by holding the right trigger when turning on the unit if needed.
The boot menu will present you with a number of options that can be used to control how the OpenPandora boots, the default is to boot the NAND as normal. You can also boot off SD’s, launch a U-Boot console over serial, and if we get the time, a console over USB serial. From within the menu the DPad and ‘B’ on the game buttons are used to select options.
Should you wish you can also bypass the bootmenu all together and boot your kernel (and ROOTFS) etc. off SD cards without any intervention by adding a suitable autoboot.txt boot script to the 1st FAT or EXT2/3 partition of SD1.
This is ideal to run alternative OS’es on the device without any intervention. It is expected that alternative distributions will come with these scripts pre-configured and ready to go, we will document this in more detail soon. If you want to try several OS’es on one SD that is even possible using the boot menu and scripts.
Note: Boot scripts.
The OpenPandora version of U-Boot supports a number of boot scripts.
- Automatically run.
autoboot.txt: This is a text file containing the raw commands you wish U-Boot to run. No error checking or checksums.
When running U-Boot will look for the presence of this script on 1st partition of SD1 providing that partition is FAT or EXT2/3.
- Run from the boot menu (Hold Right Trigger when turning on).
boot.txt: This is a text file with the raw commands you want U-Boot to run. No error checking or checksums.
boot.scr: This file contains commands in the U-Boot binary format with a header and checksum.
When running the menu U-Boot will look for the presence of these scripts on any partition of SD1 providing that partition is FAT or EXT2/3.
Want to try Ubuntu or Neuvoo just setup an appropriate SD card and script, pop it in the OpenPandora and turn it on, want to go back to the stock OS just turn off, eject the card and turn on again. Nice and simple. Just remember not to eject the card when the system is running from it ;).
The addition of the bootmenu and script support make hacking about on the OpenPandora and booting your own OS’es off SD etc. about as easy as it can be. Our plan is to offer ‘SD’ versions of the stock OS for download for people who want to hack about without messing with the copy installed to the NAND.
The stock firmware NAND layout.
We spent a discordantly large amount of time playing with the NAND layout to support a balance of simplicity, robustness and ease of hackability.
The upshot of this is that the stock firmware on the OpenPandora is split into 2 volumes on the NAND, both are UBI and UBIFS volumes, one is small (10MB) containing the kernel uImage (or kernel’s if you so desire) and the other is a large volume that takes up the rest of the NAND with the ROOTFS.
Why did we go for 2 UBIFS volumes? Simple really, mounting UBI volumes from U-Boot is a linear process, the bigger the volume the longer it takes to mount. If we mount the big ROOTFS volume directly it takes around 8 seconds, by using a separate smaller volume to hold just the kernel we get this part down to less then a second.
Storing the kernel uImage on a UBI volume allows people to play with kernel updates and modifications without resorting to flashing the kernel to RAW NAND every time (you can even symlink kernels within the UBIFS volume so try new versions and roll back if bad etc. etc.). Believe me, when your playing with the NAND layout you come to appreciate things like this.
For those that may be interested the large volume is running with in kernel LZO compression and the images are pre-made with a combination of ZLib and LZO compression to try and achieve a good combination of compression and performance (-x favor_lzo -X 20 is what we settled on for our mkubifs if there are any UBIFS geeks out there).
Note: We experimented with a number of rather more exotic file system setups 😉 and for a long while we used a combination of SquashFS as a read only root and a UBIFS read/write area joined using AUFS to provide the union root file system.
This would have given us some very nice things like the ability to totally factory reset without reflashing etc. but the cost in terms of configuration and lost NAND space when doing updates (you have the original file in the R/O SquashFS volume and a complete new copy in the R/W UBIFS area) led to this being dropped in favor of the above layout. Not to mention that the big UBIFS ROOTFS volume is a little faster.
I suspect this post may lead to more questions than it answers so please fire away via the comments if you want to and I will try and update the post.