Howto optimize Linux – SD card as rootfs

The write performance of a SD card is a little bit different then using a conventional spinning disk under Linux. The main problem is that a SD card can not write data over existing data. So whenever it want to write something it has to delete the content of that section first. To add to that initial problem, a SD card can only delete certain sizes of blocks, commonly between 2 Mb and 16 Mb depending on the SD card. So even if you have a small write cycle of let’s say a couple of Kb’s, the SD card actually has to read everything from it erasure block into memory, delete that section and then write back the altered section back onto the SD card.

Knowing this, we might want to do the following two things;

  1. Minimize any write cycles to the SD card
  2. Align the partition to the sice of the SD card his erasure block to prevent it sometimes have to delete/write two of those for most files.

Now let’s start with point 2 first as we might want to do that even before we start installing our linux system. We first need to investigate what the erasure clock size is of our SD card. The Linaro people wrote a small to to investigate that; “FlashBench“.

Download the tool, compile and run.

sudo ./flashbench -a /dev/mmcblk0  –blocksize=1024

align 134217728 pre 735µs       on 1.08ms       post 780µs      diff 324µs
align 67108864  pre 736µs       on 1.05ms       post 763µs      diff 300µs
align 33554432  pre 722µs       on 1.04ms       post 763µs      diff 294µs
align 16777216  pre 727µs       on 1.05ms       post 772µs      diff 302µs
align 8388608   pre 724µs       on 1.04ms       post 768µs      diff 299µs
align 4194304   pre 741µs       on 1.08ms       post 788µs      diff 317µs
align 2097152   pre 745µs       on 950µs        post 811µs      diff 171µs
align 1048576   pre 745µs       on 945µs        post 807µs      diff 169µs
align 524288    pre 743µs       on 936µs        post 799µs      diff 165µs
align 262144    pre 746µs       on 948µs        post 809µs      diff 171µs
align 131072    pre 737µs       on 935µs        post 804µs      diff 165µs
align 65536     pre 735µs       on 925µs        post 796µs      diff 159µs
align 32768     pre 735µs       on 925µs        post 800µs      diff 157µs
align 16384     pre 745µs       on 911µs        post 781µs      diff 148µs
align 8192      pre 785µs       on 808µs        post 725µs      diff 53.3µs
align 4096      pre 784µs       on 788µs        post 779µs      diff 5.85µs
align 2048      pre 787µs       on 793µs        post 789µs      diff 4.65µs

As you can see above there is big jump in “diff” time (the last column) from 4191304 (4mb) and 2097152 (2mb). It almost split by half. Based in the example readme in flashbench, this indicates that there was no performance overhead reading two blocks over the 2mb boundary, but there was for 4mb boundary. The guess is then that the erasure block is 4mb large on this sd-card.

Similarly, both 8 KB and 4 KB boundaries are special. The logical explanation for this is that the card has 8 KB pages, but can use multi-plane accesses to read two 8 KB pages simultaneously. So to conclude on above;

  • We want to align our partition table to the size of a 4mb erasure block.
  • We want to format it with a blocksize of 4kb and the “stride” value should be 2. This will cause ext4 to think that units of 2 blocks (8k) can and should be treated as one.
  • Ext4 should have the stripe-size set to 512. This value was calculated by taking 4M (guessed erasure block size) dividing by 8K (size of a stride, 2 times block size (4K)). This will (hopefully) cause Ext4 to try to align writes so that while erasure blocks are written continuously and make it avoid sub-block updates.

Simple, isn’t it? So if we want to start our partition aligned with a 4 mb erasure block size. Fdisk uses blocks of 512 bytes, so that means that we want to start at 4*1024^2/512 = 8192. If you want more partitions you can make use of multiple of that number, but keep in mind we start at 0 NOT 1. So if you want to have a FAT partition of around 16mb, we start at 8192 and we end that partition at (4*8192)-1 = 32767. Then the next partition will start at the exact start of a new erasure block again at 32768.

So now we want to format our partition with EXT4 and the above mentioned parameters. Let’s also disable the journalling to increase the performance using point 1 (minimise write cycles).

mkfs.ext4 -O ^has_journal -E stride=2,stripe-width=512 -b 4096 -L SDCardLinux  /dev/mmcblk0p1

That would finalise point 2, aligning the partition(s) to the size of the erasure block and optimizing the EXT4 partitioning to make full use of it.

So what can we do to minimize the amount of write cycles while running our linux distribution? Let’s first make sure that linux is no longer updating the files with their access and opening times. (when you open a file that date/time is written to the file). We can prevent that by adding the “noatime” parameters to the “/etc/fstab”.  Change it to something like this (as an example);

/dev/root / ext4 defaults,noatime 0 0

Secondly, we can chance the disk sheduler. The standard “CFQ” sheduler is optimized for spinning disk and tries to rearrange cycles such a way access times are shortened. In the case of a SD card access time are already multifold times faster then a conventional spinng disk, so let’s make sure al cycles are done immediately. We can change the sheduler in three different ways.

  1. Add linux boot parameter “elevator=noop” as kernel parameter
  2. By adding block/sda/queue/scheduler = noop to your /etc/sysfs.conf (requires the sysutils package)
  3. Compile the kernel using the “noop” sheduler as default. (This is the one I personally use now)

Now we can also move some commonly used linux folders to a tmpfs, especially the ones which we don’t care about to much, such as “tmp”. So let’s edit our “/etc/fstab” again and let add the following;

tmpfs    /var/log    tmpfs    defaults    0 0

tmpfs    /tmp        tmpfs    defaults    0 0

tmpfs    /var/tmp    tmpfs    defaults    0 0

With above tweaks you should observe some speed optimization, but at least you minimize the wear of your SD card.

Peter Steenbergen
Volg me

Peter Steenbergen

Chemical Engineer at Well Services Group
Geboren op 25 Juli in 1978 in Leiderdorp. Een techy nerd die al 15+ werkzaam is in de Olie en gas industrie. Daardoor al veel van de wereld mogen zien en meemaken. Zijn sterke affiniteit met ICT-Matters heeft hem al veel bekendheid als wel mooie projecten opgeleverd. Daarmee ook mede oprichter van "The Little Black Box", een multimedia streamer gebaseerd op XBMC/Kodi.
Peter Steenbergen
Volg me

Latest posts by Peter Steenbergen (see all)

Submit a comment