Speed up your Pi by booting to a USB flash drive

Raspberry Pi USB flash drive

Intro

If you’ve read some of the other articles on the site, you may have gathered that I have three RasPis doing useful things around my home:

  • Pi #1 is running Raspbmc, an XBMC media centre port for the Raspberry Pi.
  • Pi #2 is running this website plus an email server.
  • Pi #3 is running the Dropbox replacement OwnCloud.

All three of these ran reasonably quickly out of the box, but because the Pi is such a low powered device, every little performance boost helps.

Having said that, this is no small improvement, and the performance gain is instantly apparent.

To Business

Raspbmc’s installer has a built-in boot to SD card function – the first time you power it on, if you have a USB plugged in it will automatically set things up for you. So #1 is taken care of. That leaves #2 and #3, for which the solution is the same (the solution applies to anything running Raspbian, and may work for some of the other Pi distributions too, although I haven’t checked).

The first thing to understand here is that a Raspberry Pi is currently incapable of booting without an SD card. So although you can move your filesystem to a USB, you still need to keep the SD card plugged in.

A normal Raspbian image contains two partitions. One is the boot partition and the other is the root filesystem. The boot partition contains all the bits that your Pi looks for when it starts up, and the root filesystem contains all of the programs that you run once the system is up and running, organised in the familiar filesystem hierarchy.

What we are about to do is tell the Raspberry Pi that the root filesystem is on a USB stick, and then delete the old root on the SD card, leaving only the boot partition.

I’m going to assume that you have already made changes to your Pi that you would like to keep, and are not starting from scratch, as I doubt many people will have been foresighted enough to know that they would like to do this when they started tinkering (I certainly wasn’t!).

All of these modifications are done on a Linux laptop or desktop (the Pi is off for the duration).

Step 1 – back up your SD card

Make a note of which drive your SD card is (you can check this in your partition manager).

Now to make an image of your card and save it to your hard drive, run the following command:

sudo dd bs=4M if=/dev/sdb of=~/backup.img

(Where /dev/sdb is your SD card)

This should take about 5 minutes, just be patient.

Step 2 – copy the image over onto the USB stick

Once you have a backup image, you can insert your USB drive and copy the image over.

sudo dd bs=4M if=~/backup.img of=/dev/sdc

(where /dev/sdc is your USB stick – again, you can check this in your partition manager)

NB: we could have copied directly from the SD card to the USB stick, but it’s useful to have a backup just in case you mess it up, or your drive gets corrupted at a later date.

Step 3 – remove the partitions you don’t need

You should now have one SD card and one USB drive with identical content. You don’t need the boot partition on the USB drive, and you don’t need the root filesystem on the SD card. You don’t have to do this, but you can open your partition manager and delete those partitions to keep things neater (this will also stop you from getting confused about which is the one in use).

You may find it easier if you remove the SD card to edit the USB drive and vice versa, it’s easy to mix the two storage devices up.

Now is also a good time to resize the root filesystem on the USB drive using your partition manager. Extend it to use the free space at the end of the disk, but leave 10 MiB free at the end. I’ll explain why later.

Step 4 – work out how you are going to identify your root filesystem

Now all you need to do is tell the RasPi where the root filesystem is. This is done in a file called cmdline.txt.

The key parameter is root=/dev/mmcblk0p2. If you are never going to plug another USB device into the Pi, then you can change this to root=/dev/sda2 (the second partition on /dev/sda). Note that even if you deleted the first partition, the second will still identify itself as /dev/sdX2 unless you create a new partition table, so stick with /dev/sda2.

BUT, if you think you might use another USB device with the Pi, then you may run into problems. You might think that the RasPi always assigns the top USB /dev/sda and the bottom USB /dev/sdb… not so. Which device gets which slot is random, so /dev/sda2 might not be the partition you think it is if more than one USB device is attached.

Usually, on Ubuntu and other Linux distributions, you can refer to USB drives by their Universally Unique ID (UUID). Undortunately, the Raspberry Pi bootloader isn’t as sophisticated as Ubuntu’s GRUB. I’ve tried assigning the root location by UUID, but it doesn’t work.

However, I did manage to find a solution here. While you can’t use the UUID, you can use the PARTUUID.

First, you need to convert your partition table from DOS to GPT. To do this, you’ll need the commandline tool gdisk. * If you didn’t leave that extra 10MiB at the end of the disk, you’ll get an error at this stage about the partition table and a partition overlapping.

sudo apt-get update
sudo apt-get install gdisk
gdisk /dev/sdb

(where /dev/sdb is the usb drive).

You will now be greeted with a prompt. Type ? and press enter to list your options.

Now choose “recovery and transformation options (experts only)”.

Again, type ? and then choose “load MBR and build fresh GPT from it”.

Finally, type ? and choose “write table to disk and exit”.

Now take the USB drive out, re-insert it, and run gdisk again:

sudo gdisk /dev/sdb

Now press “i”, and then “2″ to display detailed information on the second partition on the drive /dev/sdb2. You should get output like this:

Partition number (1-2): 2
Partition GUID code: 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem)
Partition unique GUID: CA9D1961-4627-46BB-BACA-7C7106977E5C
First sector: 122880 (at 60.0 MiB)
Last sector: 60545023 (at 28.9 GiB)
Partition size: 60422144 sectors (28.8 GiB)
Attribute flags: 0000000000000000
Partition name: 'Linux filesystem'

The “Partition unique GUID” is what we’re after. Paste it into a text editor for now, you’ll need it later.

Step 5 – edit the boot partition on the SD card

Mount the boot partition on the card, locate the partition, and then edit cmdline.txt.

We need to change root=/dev/mmcblk0p2 to root=PARTUUID=XXXXXX (fill in your Partition Unique GUID in the place of XXXXXX).

cd /media/boot
ls
sudo nano cmdline.txt

Press CTRL+X when you’re done, and say yes when prompted to save.

Close the terminal window and unmount the USB drive.

Step 6 – Boot your Pi to the USB drive

Now all that’s left is to insert your SD card, your USB drive, and reconnect the power cable.

If all goes well you should boot into a much speedier system, with greater stability and reliability.

Important note!

I wrote this tutorial in October 2013, and was running an email server and a WordPress website on the Pi from then until March 2014... and then the flash drive bricked and I lost everything because I didn't have a backup!

So, just remember: USB flash drives are faster and more reliable than SD cards, but they aren't really meant to be used for this kind of application. As a result, you should keep regular backups!

The website I was running was generating 1100 visits and about 10k hits every day - imagine how different that is to the occasional reads and writes that the USB drives are designed for. If you want a more reliable server, you can use a self powered HDD and use this tutorial to put the root filesystem on that.

Category: 
Type: 

Comments

Hi Sam,

Thanks for a great tutorial, and for sharing your expertise.

I've run into a problem copying the SD card image to a supposedly same sized USB stick. Although both are quoted as 16GB, the stick is marginally smaller than the card and the image won't copy (error writing '/dev/sdc': No space left on the device)

I've tried using GParted to resize the SD card's filesystem partition (only 3.6GB are actually being used) but the image created by sudo dd bs=4M if=/dev/sdb of=~/backup.img remains the same size (i.e. the full capacity of the SD)

Is there any way of sizing down the image being copied copy? Or maybe just copying the filesystem partition? (since the boot partition is to remain on the SD card anyway)

Thanks,

Jerome

Yep, you should be able to do something like:

 sudo dd bs=4M if=/dev/sdb2 of=~/backup.img

...assuming sdb2 is the root filesystem partition on the SD card. You have exactly the right idea, it's best to reduce the size of the partition on the SD card before making the images if you can (dd is so slow!).

When I did mine I was using a 4GB SD card so it wouldn't have made a huge difference (it was mostly full) but you certainly don't want to wait for your box to copy an extra 8GB of nothing :)

Sam

Hi,
what can I do when my SD card is 16 GB of storage and my USB stick only 4 GB?
The image I made some time ago usind dd has a size if about 15 GB.
Of course the real data on the SD card are less than 4 GB in size.
Can I resize the whole data on SD to occupy less than 4 GB before I create a dd image?

Thanks

Yep, best to do it before you make the image. Try the program gparted if you want a graphical way - you can just grab the end of the partition and drag it left to shrink its size.

Sam

Hi Sam,

Thanks for your reply!
Well, I finaly decided to do that a bit differently.
As I told you, I wanted to migrate my raspi (raspbian) from a 16GB SD card to a combination 8GB SD card plus 8 GB USB stick.
So I used a Fedora 20 VMWarePlayer guest machine (where parted understands ext4) and created an ext4 <8GB primary partition on
that USB stick.
Then I connected the original SD card to the Fedora machine and created a tar ball of the SD card's / file system with the options
-p --ignore-failed-read --same-owner (even if I wasn't sure about the real necesity).
I extracted that tar ball then onto the USB stick.
In this new / file system I replaced in the /etc/fstab the second SD card's partition device name (/dev/mmcblk0p2) with /dev/sda1.
With just one USB stick connected on boot it seems to work securely.
I also tried UUID and PARTUUID in the cmdline.txt but neither worked.
Next I created a boot partition on the new 8GB SD card with fat16 file system (with parted) which was a bit bigger than that on the old SD card (well and the rest as an ext4 partition).
Then I created a dd image from the original boot partition, which I stored on the USB stick and again with the dd cmd I copied it onto the
new SD card's boot partition.
Then I just had to change the cmdline.txt of the new boot partition to show to the USB stick as the rootfs (=/dev/sda1).

Nice work man! I have recovered all my data but i have got some difficulties because this was my first time and i am sure these will not be there if i unluckily deletes my precious data :)

Hi Sam,
You have one of the most useful websites i have ever been in that is all about the Pi :)

i was wondering if i can save some money and combine the owncloud with the openelec(kodi)???
any thoughts?

Hi,

Thanks! I'm happy you're getting some good use out of it.

I wouldn't recommend it, kodi and owncloud are both quite resource intensive apps, and kodi makes certain assumptions about only being accessible on the LAN (like having a remote served on the http port) that might not play well with apache set up for owncloud. Possible, but it's cleaner to separate the two.

Sam

Hi,

The instructions are for Linux, not Mac. Windows doesn't understand some of the file formats, and it doesn't have these utilities, so you can't use a Windows box for this directly.

What you could do is create a live disk or USB stick of Ubuntu, boot into that on your computer, and then follow the tutorial.

Sam

Sam Thanks for your step by step guide.
I have done things a bit backwards, with your help i now have a working email server with fail2ban it's blocked 4 repeat offenders in just 9 days.
now looking too add a hard drive, but i only my pi and a windows laptop.
found this. it didn't work straight out of the box as formatting the disc complained and the script exited, but once that was fixed the script showed how to copy on the pi with the (rsync -ax / /mnt) command, hope this makes scene.
Script came from https://learn.adafruit.com/external-drive-as-raspberry-pi-root

Hello Sam.

I followed your instructions, but on bootup the pi freezes. I get the following error: "end Kernel panic - not syncing: No working init found. Try passing init= option to kernel. See Linux Documentation/init.txt for guidance".

I would expect this to be a typo in the GUID or something like that - if the disk isn't mounted, then none of the system files will be found.

Worth double checking!

Sam

Hello again

That doesn't appear to be the issue. It begins loading on the USB, then immediately stops and outputs the error. Any suggestions?

What makes you think that's not it? The init files are stored in the root filesystem, which is on the USB drive.

Do you have another machine running Linux? Can you use it to cat your cmdline.txt and print the output of the gdisk commands?

The only other thing I can think of is that you could be using a passport style HDD that isn't self-powered (the pi doesn't output enough power to reliably power drives like that) - this would stop it being mounted successfully, preventing the init system from being loaded.

Sam

Followed the advice re booting the R-Pi to USB

When booted up if I leave the USB keyboard and mouse plugged in to the Pi everything is painfully slow, if I disconnect these devices and access via network using VNC it is fine.

Tried a few times just in case but need to disconnect mouse and KB.

Geoff

Hi,

It could be a bad power supply - not enough juice to power all of the bits at once?

Sam

I know the new Pi doesn't have all the improvements you had hoped for (still USB 2), but have you used it? Do all the instructions on this page still apply? I just got one for the first time and plan to make your posts my go-to source. I just want to make sure raspberries are still raspberries. Ha. Did you see how I did that? Oh, sorry. Anyway, just want to make sure everything is still applicable so my Newbie self doesn't get frustrated. Thanks.

Hi,

Yes the tutorials should still work, but let me know if you find anything different. I have used Pi A, Pi B and Pi 2 for various bits and pieces.

Sam

hi sam.
I have read and understand your tutorial. But in my case i used the one from adafruit and it have a reference to yours (wished i used yours).
I have a problem. i am getting the color splash screen and that is i have already put on the cmdline on my sd to point to it self for boot but is not working got same response, my sd sames to be working i can read it on mu ubuntu machine whit no problem. my usb stick readable whit out problems. I am guessing the problem is the sd but because is readable i have my doubts. Any input.
And another question on the end of your tutorial you said that you had a problem and no backup. Can i just copy the all of the usb to my ubuntu machine or can i made a image copy like in my sd card? thank you for all of your help

Hi,

Unfortunately I don't know how to help you with someone else's tutorial.

For the backups, the best way to do afull backup is to make an image.

Sam

Hi Sam,
New year coming soon. Here wish you have a great Christmas. Really appreciate your tutorial.
I got a problem on my disk. I didn't follow your instruction to remain 10 MiB.(I guess I miss this important direction.) Then, following the sequence steps, I couldn't install any packages well while I restart the raspberry pi.(Amazing!!!) So, I intend to format my root partition on my usb drive and make a new one for restarting again. However, I am struggling how to format the disk and get a information about root partition owing a mark-protective MBR which is unfamiliar to me. I google it and get known about it. Although I did't know why I couldn't install any packages, but as long as I clean the disk, I can make my disk drive work again. Yet I didn't know how to refresh my disk drive.(I had attempted to format it by any tools on Windows, MAC, and Ubuntu.) Do you know any methods to solve this problem? Also, I wonder why I couldn't install any packages.

Best regards,
Jeff

Great tutorial, thank you very much! Gonna update my 3 Pi's with this technique, tired of the SD corruption...

You can update the firmware or OTP bin (PROM) on the rpi3 so that it will boot from a usb if it finds the SD is empty.
Plus you can boot the other rpi's by putting a cheap SD with only the bootcode.bin file on the card.
https://www.raspberrypi.org/documentation/hardware/raspberrypi/bootmodes/

If you want to copy your existing files to the usb you can use rsync to make it easy. I used this tutorial...
http://www.makeuseof.com/tag/make-raspberry-pi-3-boot-usb/

Pages

Add new comment