If you would like to install extra packages on OpenWrt, but you have run out of space on your router’s internal flash memory, then this tutorial is for you. The plan is to copy the OpenWrt’s root filesystem onto an external USB flash drive, and tell the router to switch to that when it boots up. All you need is a standard USB flash drive, a USB capable router running OpenWrt, and about 30 mins.
The OpenWrt wiki pages for this subject are very good. If you’re interested in reading about all the different options, then take a look (for example, it’s possible to mount the USB drive on /opt and install extras here, but some packages expect to be installed in root and won’t behave nicely). In case you’d like to look it up later, the type of extroot that I’m talking about here is external root (aka pivot root), not external overlay (aka pivot-overlay).
This assumes that you have already formatted your external USB drive with a journaling filesystem (like EXT3 or EXT4). If you haven’t, do this first (the partition managers in Ubuntu and Kubuntu do this very easily, do a google search for a tutorial if you’re unsure how to do this). To use pivot root, you must be using a version of OpenWrt that is newer than v12 (so 12.09 Attitude Adjustment is fine, but Backfire is not). First, connect to OpenWrt via ssh or telnet.
Install some packages
First, install the package that will swap your root from the router’s inbuilt flash memory to the external USB flash device:
opkg update opkg install block-mount
Copy your current root filesystem to the USB flash drive
Before you do this, make sure your router is secure (e.g. SSH login isn’t possible from WAN, wifi interfaces are password protected etc). This is important because if something goes wrong when your router boots up then OpenWrt will be unable to boot to the USB flash drive, leaving you with whatever your present setup is until you can make it boot to the USB drive properly. First, install fdisk, a tool to tell you info about attached devices:
opkg update opkg install fdisk
Now, insert your USB drive and run the following command to get some details about it:
root@OpenWrt:~# fdisk -l ... Disk /dev/sda: 4229 MB, 4229496832 bytes 255 heads, 63 sectors/track, 514 cylinders, total 8260736 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00933453 Device Boot Start End Blocks Id System /dev/sda1 63 8257409 4128673+ 83 Linux
You see your USB drive listed under “Device”. Now make a mount point for the drive. You can choose whatever you like instead of sda1, just remember what you chose later!
Mount the drive on the mount point you just created (replace /dev/sda2 with whatever you got from the fdisk command, and /mnt/sda1 with whatever you chose as your mount point):
mount /dev/sda2 /mnt/sda1
Now copy the root filesystem from the router’s inbuilt flash to the USB drive with these commands (if you chose a name other than /mnt/sda1 in the previous step, then replace /mnt/sda1 in line 3):
mkdir -p /tmp/cproot mount --bind / /tmp/cproot tar -C /tmp/cproot -cvf - . | tar -C /mnt/sda1 -xf - umount /tmp/cproot
You now have a USB drive with a copy of your router’s filesystem on it. The next step is to make it automatically mount when you boot up, and use it as root.
Open /etc/config/fstab with your favourite text editor:
If nano is not installed, you can use vi (which is installed by default, but is horrible!) or you can just install nano with:
opkg update opkg install nano
Add the following to the file:
config mount option target / option device /dev/sda1 option fstype ext3 option options rw,sync option enabled 1 option enabled_fsck 0
Reboot and Check
When you reboot, you should now be running OpenWrt from your USB stick. Check your current mounts with this command:
Output should be something similar to the following:
root@OpenWrt:~# mount rootfs on / type rootfs (rw) /dev/root on /rom type squashfs (ro,relatime) proc on /proc type proc (rw,noatime) sysfs on /sys type sysfs (rw,noatime) tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime,size=63340k) tmpfs on /dev type tmpfs (rw,noatime,size=512k,mode=755) devpts on /dev/pts type devpts (rw,noatime,mode=600) /dev/sda1 on / type ext4 (rw,sync,relatime,user_xattr,barrier=1,data=ordered) debugfs on /sys/kernel/debug type debugfs (rw,relatime) none on /proc/bus/usb type usbfs (rw,relatime)
See how much space you’ve created for exciting packages with this command:
Which should show you something like this:
root@OpenWrt:~# df -h Filesystem Size Used Available Use% Mounted on rootfs 3.9G 157.1M 3.6G 4% / /dev/root 2.0M 2.0M 0 100% /rom tmpfs 61.9M 932.0K 60.9M 1% /tmp tmpfs 512.0K 0 512.0K 0% /dev /dev/sda1 3.9G 157.1M 3.6G 4% /
You’re all done. Have fun building your personalised OpenWrt cathedral!
Thank you for the great article. I wanted to add some things that I had to do extra on my router TP-LINK MR3040, that may need to be added to the article in order for it to be more generic.
1. I had to install via opkg the package kmod-scsi-core, because otherwise, even though I saw the USB device using lsusb, the ext4 device was nowhere to be seen when doing fdisk -l.
2. My /etc/config/fstab file was completly empty and after adding the whole "config mount", after reboot nothing happened. What I did in the end was run #block detect and copy the "config global" part into /etc/config/fstab above the "config mount" part
3. This is what I now run in order to install everything I need:
opkg install kmod-scsi-core kmod-usb-core kmod-usb2 kmod-usb-storage kmod-fs-ext4 fdisk block-mount
I'm trying to follow your advice however the symlinks are not being copied:
tar: can't create symlink from ./etc/fstab to /tmp/fstab: Operation not permitted
Is it important?
The Arduino Yun (running OpenWRT) has a nice tool for expanding the disk size: http://arduino.cc/en/Tutorial/ExpandingYunDiskSpace
Looking inside it for the console commands, it formats the new disk to ext4 (mkfs.ext4) and then copies the files with rsync.
Maybe it'll be helpful for someone who gets to this page...
worked like a charm, i followed a few other guides with no luck, works well, should i create a swap partition on sda1 or leave with no swap, thanks again for the efforts here and time spent creating this tutorial
EXCELLENT guide! Thx!
But... I DO need a swap partition, because transmission-daemon is eating my memory so fast and stops itself after a while... I'm running OpenWRT on TL-WR710N; I can't mount swap partition, whatever I do. I get only this message: "block: failed to swapon /dev/sda5". Do you have any advice?
Thank you very much !
Your tutorial worked fine ! Now I able to install openvpn ! and make a backup of the entire router system !
I have a TP-Link 3420 (HW 1.1) with OpenWrt Chaos Calmer 15.05 / LuCI (git-15.248.30277-3836b45)
I'm trying to follow your instructions but, after installing block-mount, I cannot install fdisk because of lack of free space.
So i hoped to use
opkg install fdisk -d USB_path but I cannot mount the USB device (/dev/sda1) because my filesystem is read-only.
rootfs on / type rootfs (rw)
/dev/root on /rom type squashfs (ro,relatime)
proc on /proc type proc (rw,noatime)
sysfs on /sys type sysfs (rw,noatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime)
/dev/mtdblock3 on /overlay type jffs2 (rw,noatime)
overlayfs:/overlay on / type overlay (ro,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)
tmpfs on /dev type tmpfs (rw,relatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,relatime,mode=600)
debugfs on /sys/kernel/debug type debugfs (rw,noatime)
Hi again SAM!
I tried installing fdisk in ram..
opkg install fdisk -d ram
Installing fdisk (2.25.2-4) to ram...
Installing libblkid (2.25.2-4) to ram...
Installing libuuid (2.25.2-4) to ram...
Installing libsmartcols (2.25.2-4) to ram...
grep: /usr/lib/opkg/info/libuuid.control: No such file or directory
cat: can't open '/usr/lib/opkg/info/libuuid.list': No such file or directory
cat: can't open '/usr/lib/opkg/info/libblkid.list': No such file or directory
grep: /usr/lib/opkg/info/libsmartcols.control: No such file or directory
cat: can't open '/usr/lib/opkg/info/libsmartcols.list': No such file or directory
grep: /usr/lib/opkg/info/fdisk.control: No such file or directory
cat: can't open '/usr/lib/opkg/info/fdisk.list': No such file or directory
mountoutput again, I saw this:
overlayfs:/overlay on / type overlay (ro,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work)If you look on the OpenWrt wiki page describing flash layout under the "mount points" section, it says that the purpose of
/overlayis to contain the writable part of the filesystem, so I have no idea why it's read only on yours!
"/overlay" is the writable part of the file system that gets merged with /rom to create a uniform /-tree. It contains anything that was written to the router after installation, e.g. changed configuration files, additional packages installed with OPKG, etc. It is formated with JFFS2.Anyway, I guess we can just try re-mounting it read-writable! I'm no
mountguru, but I think this will work:
mount -o remount,rw /overlayI think mount reads the default options in /etc/fstab and merges them with whatever you specify on the commandline, so I don't think it's necessary to list all the options (lowerdir, upperdir etc.) again, but I'm not certain. Finally, are you sure you want to remove
wpad-mini? Seems like an important package. If it was me I think I would remove some luci packages temporarily (so you'd lose the web interface) because they're not required for the router to function, and you can still interact with it via SSH. Sam
Thanks for your reply !
I tried removing some package (odhcp6c and wpad-mini, for example) with
opkg remove --force-remove but nothing happens.
Removing package wpad-mini from root...
but nothing happens (df -h has the same values..). I believe it's about READ-ONLY filesystem..
This is the opkg -list-installed output.
base-files - 157-r46767
block-mount - 2015-05-24-09027fc86babc3986027a0e677aca1b6999a9e14
busybox - 1.23.2-1
dnsmasq - 2.73-1
dropbear - 2015.67-1
firewall - 2015-07-27
fstools - 2015-05-24-09027fc86babc3986027a0e677aca1b6999a9e14
hostapd-common - 2015-03-25-1
ip6tables - 1.4.21-1
iptables - 1.4.21-1
iw - 3.17-1
jshn - 2015-06-14-d1c66ef1131d14f0ed197b368d03f71b964e45f8
jsonfilter - 2014-06-19-cdc760c58077f44fc40adbbe41e1556a67c1b9a9
kernel - 3.18.20-1-7bed08fa9c06eb8089e82c200340ec66
kmod-ath - 3.18.20+2015-03-09-3
kmod-ath9k - 3.18.20+2015-03-09-3
kmod-ath9k-common - 3.18.20+2015-03-09-3
kmod-cfg80211 - 3.18.20+2015-03-09-3
kmod-crypto-aes - 3.18.20-1
kmod-crypto-arc4 - 3.18.20-1
kmod-crypto-core - 3.18.20-1
kmod-gpio-button-hotplug - 3.18.20-1
kmod-ip6tables - 3.18.20-1
kmod-ipt-conntrack - 3.18.20-1
kmod-ipt-core - 3.18.20-1
kmod-ipt-nat - 3.18.20-1
kmod-ipv6 - 3.18.20-1
kmod-ledtrig-usbdev - 3.18.20-1
kmod-lib-crc-ccitt - 3.18.20-1
kmod-mac80211 - 3.18.20+2015-03-09-3
kmod-nf-conntrack - 3.18.20-1
kmod-nf-conntrack6 - 3.18.20-1
kmod-nf-ipt - 3.18.20-1
kmod-nf-ipt6 - 3.18.20-1
kmod-nf-nat - 3.18.20-1
kmod-nf-nathelper - 3.18.20-1
kmod-nls-base - 3.18.20-1
kmod-ppp - 3.18.20-1
kmod-pppoe - 3.18.20-1
kmod-pppox - 3.18.20-1
kmod-scsi-core - 3.18.20-1
kmod-slhc - 3.18.20-1
kmod-usb-core - 3.18.20-1
kmod-usb-storage - 3.18.20-1
kmod-usb-storage-extras - 3.18.20-1
kmod-usb2 - 3.18.20-1
libblobmsg-json - 2015-06-14-d1c66ef1131d14f0ed197b368d03f71b964e45f8
libc - 0.9.33.2-1
libgcc - 4.8-linaro-1
libip4tc - 1.4.21-1
libip6tc - 1.4.21-1
libiwinfo - 2015-06-01-ade8b1b299cbd5748db1acf80dd3e9f567938371
libiwinfo-lua - 2015-06-01-ade8b1b299cbd5748db1acf80dd3e9f567938371
libjson-c - 0.12-1
libjson-script - 2015-06-14-d1c66ef1131d14f0ed197b368d03f71b964e45f8
liblua - 5.1.5-1
libnl-tiny - 0.1-4
libpcre - 8.37-2
libubox - 2015-06-14-d1c66ef1131d14f0ed197b368d03f71b964e45f8
libubus - 2015-05-25-f361bfa5fcb2daadf3b160583ce665024f8d108e
libubus-lua - 2015-05-25-f361bfa5fcb2daadf3b160583ce665024f8d108e
libuci - 2015-04-09.1-1
libuci-lua - 2015-04-09.1-1
libxtables - 1.4.21-1
lua - 5.1.5-1
luci - git-15.248.30277-3836b45-1
luci-app-firewall - git-15.248.30277-3836b45-1
luci-base - git-15.248.30277-3836b45-1
luci-lib-ip - git-15.248.30277-3836b45-1
luci-lib-nixio - git-15.248.30277-3836b45-1
luci-mod-admin-full - git-15.248.30277-3836b45-1
luci-proto-ipv6 - git-15.248.30277-3836b45-1
luci-proto-ppp - git-15.248.30277-3836b45-1
luci-theme-bootstrap - git-15.248.30277-3836b45-1
mtd - 21
netifd - 2015-06-08-8795f9ef89626cd658f615c78c6a17e990c0dcaa
odhcp6c - 2015-07-13-024525798c5f6aba3af9b2ef7b3af2f3c14f1db8
odhcpd - 2015-05-21-2ebf6c8216287983779c8ec6597d30893b914a7c
opkg - 9c97d5ecd795709c8584e972bfdf3aee3a5b846d-7
ppp - 2.4.7-6
ppp-mod-pppoe - 2.4.7-6
procd - 2015-08-16-0da5bf2ff222d1a499172a6e09507388676b5a08
rpcd - 2015-05-17-3d655417ab44d93aad56a6d4a668daf24b127b84
swconfig - 10
uboot-envtools - 2014.10-2
ubox - 2015-07-14-907d046c8929fb74e5a3502a9498198695e62ad8
ubus - 2015-05-25-f361bfa5fcb2daadf3b160583ce665024f8d108e
ubusd - 2015-05-25-f361bfa5fcb2daadf3b160583ce665024f8d108e
uci - 2015-04-09.1-1
uhttpd - 2015-08-17-f91788b809d9726126e9cf4384fedbbb0c5b8a73
uhttpd-mod-ubus - 2015-08-17-f91788b809d9726126e9cf4384fedbbb0c5b8a73
usign - 2015-05-08-cf8dcdb8a4e874c77f3e9a8e9b643e8c17b19131
wpad-mini - 2015-03-25-1
Thanks for any other help.
Thanks this opens a world of possibilities! Although I had to build my own minimal image without luci, then follow your steps and then install luci and everything else.
Thank you so much for the great tutorial! Your tutorial just made my $12 toy something I can actually use.
Out of the box (so to speak) a fresh firmware flash on the VoCore2-Lite leaves you with a whopping 916.0K of available space on root:
Filesystem Size Used Available Use% Mounted on
rootfs 1.1M 236.0K 916.0K 20% /
After formatting a 16GB USB flashdrive and following your tutorial:
Filesystem Size Used Available Use% Mounted on
rootfs 14.1G 57.9M 13.3G 0% /
Now I can actually install more then three packages without running out of space! Thanks!
I've LG01 gateway runs openWrt system. I want to attach USB flash device for that i followed your article and https://openwrt.org/docs/guide-user/storage/usb-drives-quickstart artical.
Now USB flash device is mounted.
root@dragino-1c0b14:~# df -h
Filesystem Size Used Available Use% Mounted on
rootfs 3.5M 632.0K 2.9M 18% /
/dev/root 11.0M 11.0M 0 100% /rom
tmpfs 29.8M 484.0K 29.4M 2% /tmp
/dev/mtdblock4 3.5M 632.0K 2.9M 18% /overlay
overlayfs:/overlay 3.5M 632.0K 2.9M 18% /
tmpfs 512.0K 0 512.0K 0% /dev
/dev/sda1 7.0G 16.3M 6.6G 0% /mnt/sda1
now i wanted to copy some file to USB flash device. How to do it, which place i've to store it and run .
Can you update the howto for OpenWRT 18.06.4?
I get the warning "block-mount is deprecated".
And it triggered a boot loop.
Thank you for writing the guide. By following your guide a was able to move OpenWrt’s root filesystem to a old external 4GB USB flash drive. Afterwards I was able to install all of the package updates.
You did not mention if there should be any data in the /etc/config/fstab file; I had the information listed below so I left the current into and added the new lines after the current data. The data that I added was not present after I restarted the router and booted from the USB flash drive.
option anon_swap '0'
option anon_mount '0'
option auto_swap '1'
option auto_mount '1'
option delay_root '5'
option check_fs '0'
Thank you, your guide worked for me on a Netgear WNDR3700v1 running OpenWRT 19.07.