"The lowdown on DBCache and ROM size": comment by joad

Fri, 10 Feb 2006 03:41:07 EST

Great article, thanks for the excellent explanation of this intensively troubling issue and how to possibly minimize the problems of filling up dbcache.

"The lowdown on DBCache and ROM size": comment by Calash

Thu, 9 Feb 2006 16:37:09 EST

It continues to amaze me how people can, by what can be called trial and error, discover so much about complex devices like this. I follow several forums of various devices that I own and in each one there is a group of what would be called “Hardcore” owners who tear apart the devices and learn the secrets.

My thanks to people like you, who not only take the time to learn this but also share it with the rest of us

"The lowdown on DBCache and ROM size" by matt

Tue, 7 Feb 2006 16:02:00 EST

There’s been a lot of speculation on how ROM sizes affect free space and available DB cache since Shadowmite opened up the world of ROM hacking. Many theories have been proposed, but I’d like to set the record straight by listing what we’ve learned via disassembly, trial-and-error and various tests.

The key questions are how the size of ROM affects:

the available amount of DBCache
the “free space” measure
the amount of “dynamic memory”

To properly answer these questions, it’s best to give a quick overview of what happens when a Treo boots after a reset.

The boot process

One of the key points to know is that the only persistent storage available on the Treo 650 is the internal DOC (disk-on-chip). This is a 32MB flash memory that uses a special library from MSystems to make it look like regular random-access storage.

The first thing that the CPU does is load the initial program loader or IPL from a special location on the DOC. The sole job of the IPL is to load the secondary program loader, or the SPL into memory at a given address.

The SPL has a number of responsibilities itself, but the one we’re most concerned about is the job of loading the tertiary program loader, or TPL.

The TPL is a very simple program that takes the compressed ROM stored on the DOC, uncompresses it to RAM and boots the specified image of PalmOS.

The ROM files that the TPL decompresses are placed directly into an area of memory that is reserved for the decompressed ROM files.

The DOC partition layout

The Treo’s internal storage is partitioned internally into two separate areas: the ROM/boot partition and the NVFS partition. These two partitions are essentially treated as two separate storage devices.

The only time that the partition layout changes is during a ROM update. The size of the ROM partition is actually hardcoded in the program that updates the ROM files. The size of the NVFS partition is calculated as being the remaining space available on the DOC itself. For example, if the rom updater was hard-coded to an 8MB ROM partition, the NVFS size would be approximately 24MB.

The ROM/boot partition

As previously mentioned, the ROM/boot partition contains the TPL that boots the ultimate PalmOS image. It also contains the program that is often referred to as the “HTC bootloader”, which is a separate program used for diagnostics (and ROM recovery with our tool).

The ROM/boot partition is not a standard partition format of any kind. It contains three zip files laid out at three specific locations, with the majority of the remainder of the partition containing zeros. The three zip files contained in this partition are:

The HTC bootloader
The PalmOS TPL
The ROM

The ROM zip file is an exact copy of the ROM that was uploaded during the ROM update process. There is no further compression done on this file by the ROM update process.

If the total length of the ROM/boot partition is 9MB and the files within only total 7MB, 2MB of the partition contains zero bytes and is effectively wasted.

Division of RAM

The three things that PalmOS uses the main memory for are uncompressed ROM files, DBCache and dynamic RAM.

The size of the uncompressed ROM files is a fixed number that varies only by ROM contents. The size of dynamic RAM is fixed, but varies slightly according to the version of PalmOS that is loaded. The end result of this is that DBCache is calculated as the remaining memory once the ROM is uncompress and dynamic RAM is allocated.

The conclusions

So, let’s revisit our original question:

How does the size of the ROM affect the available amount of DBCache, the “free space” measure and the amount of “dynamic memory”.

First of all, let’s look at DBCache. It was mentioned above that DBCache is calculated by taking the remaining memory once the uncompressed ROM and dynamic RAM have been allocated. The conclusion we can draw from this is that when you reduce the size of your ROM, total DBCache increases by the size of the files you remove. The same is true in reverse – adding files reduces your DBCache in the same way.

The “free space” measure is calculated by taking the size of the NVFS partition and subtracting the stored NVFS data. As mentioned above, the NVFS partition is of a fixed size given a version of the ROM update application, so adding or removing files from your ROM will not affect the total amount of free space.

Note that you can, however, use a different ROM updater application to provide a smaller ROM partition size and, therefore, a larger NVFS and “free space” amount.

Finally, we mentioned above that the amount of dynamic RAM is effectively fixed, given a certain version of firmware. This means that reducing or increasing the size of your ROM has no effect whatsoever on the remaining dynamic RAM size.

Recommendations

Based on the given data, it is recommended to make your ROM as small as possible to increase your DBCache as much as possible.

Adding files to the ROM is effectively akin to locking them into DBCache using a program such as Resco Locker.

If you can, strip out any non-essential Palm applications from your ROM and run them from RAM, if possible.

Once you have stripped your ROM, ensure that you are using a ROM updater that gives you the smallest possible ROM partition that still fits the size of your ROM.

grack.com comments on The lowdown on DBCache and ROM size