Kodi server part 3: Automounting External Drives with udev

This is the third part of a multi-part tutorial describing how to configure the "perfect" Kodi media centre running on top of ubuntu server.

Other parts of the tutorial may be found here:

This part of the tutorial sets out a few different methods I've tried for auto-mounting known external hard drives, and the working solution I arrived at. My hard drives contain the media files in my library, so I need them to be mounted at consistent locations to avoid breaking Kodi's file indexing and the NFS shares.

Automatically mount external drives on consistent mount points

This is the bit that took me the longest to figure out. I tried a couple of other solutions along the way that didn't work before finally finding a solution using udev. I'll briefly cover what didn't work, as I think it's useful to know; if you don't care and just want to know "the answer" then feel free to skip ahead.

Firstly, to clarify the problem: I have a set of external drives containing media files that I want to export via NFS. When I plug these in, they will be given a drive number by the kernel (like /dev/sdb) but they won't be mounted automatically. I need them to be mounted so that the files can be shared via NFS. Not only that, but they must be mounted at the same location each time, otherwise the file locations will change and the media library will break.

First attempt: usbmount

First, I tried a tool called usbmount. This would be great on a system where you just wanted to automatically mount all drives somewhere in /media/ so that Kodi can access the files. For example, you might have a USB stick with photos on it that you want to show on the TV, without needing the files to be in the library long term.

If you wanted to use usbmount for something else, all you need to do is install it:

sudo apt-get update
sudo apt-get install usbmount

Usbmount will automatically create a load of mount points in /media like /media/usb0, /media/usb1 etc. and use the first available mount point for removable drives when they are inserted.

This doesn't work for us because there is no guaranteed consistency in the mount points: if you have three hard drives, then a drive that is mounted at /media/usb0 may become /media/usb2 the next time the system is booted or drives are removed and re-inserted. This would break media paths in the library.

Second attempt: automount

Automount is a really neat tool that allows known drives to be automounted when they are required, an optionally unmounted after a period of inactivity.

Known drives and their mount points are defined in mapping files, and then automount monitors the filesystem under each mount point. When a user requests a file under the mount point, automount quickly mounts the filesystem.

Unfortunately, this doesn't work for our purposes because an NFS request to see files in /srv/nfs/drive1 doesn't trigger automount in the same way that a local user running a command like ls /srv/nfs/drive1 does. Nevertheless, here is the setup I used, which does correctly mount the filesystem when a local user asks for files under one of the mount points.

I installed automount:

sudo apt-get update
sudo apt-get install autofs

The default automount config file /etc/auto.master contains this line:

+dir:/etc/auto.master.d

Which means that files in /etc/auto.master.d will be included in the config. I added a file at /etc/auto.master.d/local.autofs containing the following:

/srv/nfs                /etc/auto.ext-usb --timeout=0,defaults,user,noexec

The above says to look in /etc/auto.ext-usb for mapping between disks and their predefined mount points relative to /srv/nfs, and mount them with the following options:

  • timeout=0 means that the drive will never be automatically unmounted (0=infinite, if an option like 30 was used then the drive would be unmounted if it was not used for 30 seconds).
  • defaults means use the default mount options will be used: rw, suid, dev, exec, auto, nouser, async and relatime
  • user overrides the default nouser and allows normal system users to trigger the disk to be mounted
  • noexec overrides the default exec and prevents binaries on the disk from being executed

Then I created the file /etc/auto.ext-usb with contents:

# list of drives to be mounted in /srv/nfs
drive1         -fstype=auto            :/dev/disk/by-label/BUFFALO
drive2            -fstype=auto            :/dev/disk/by-label/EXT4
drive3           -fstype=auto            :/dev/disk/by-partuuid/c5932834-a8c6-4bc2-868e-37b586debd06

You can see here that I have used the default disk labels created by udev to identify which disks to mount. Two of the disks have nice human readable labels, and for the other one I used the universally unique ID of the partition to identify what should be mounted.

After making these changes, I restarted autofs:

sudo service autofs restart

Before requesting any files under the mount point, the drives were not mounted:

$ ls -a /srv/nfs
.  ..

Asking for files inside one of the mount points caused the drive to be mounted:

$ ls /srv/nfs/drive1
Films  lost+found  Music  TV Shows

Unfortunately, nfs requests didn't trigger the mount!

After messing around with automount / autofs for a while, I realised that I would need another method to run a command as a local user to trigger the drive to be mounted. I discovered that it is possible to write a udev rule to run a command when a drive is inserted, and tried using this to run /bin/ls /srv/nfs/drive1 inside the mount point for the drive. This didn't work, I think the reason is either because the command was executed too early, or because of the weird environment that such commands are run in.

This was probably for the best, because if you can run a command when a drive is inserted then you can do away with automount and mount the drive directly with mount. If you know exactly why it didn't work though, please let me know!

Working solution: udev

Udev is a device manager, which is responsible for managing /dev/ directory and giving drives names ("nodes") like /dev/sda when they are inserted.

The default rules in /lib/udev/rules.d/ are pretty useful already, and give nodes sorted by label, UUID, kernel name (/dev/sdX) etc. for usb devices.

We need to add a new rule for each removable hard disk, which will uniquely identifiy the disk and run the right mount command. We will put the new rules in a file called /etc/udev/rules.d/99-triggerDiskMountsForNFS.rules. Here is an example rule, which I will explain shortly:

SUBSYSTEM=="block",ATTR{partition}=="1",ATTR{size}=="1953521664",ATTRS{model}=="MQ01ABD100",ATTRS{vendor}=="drive1",SYMLINK+="drive1",RUN+="/bin/mount -o defaults,user,noexec /dev/drive1 /srv/nfs/drive1"

Breaking this down into three parts, we have:

  1. SUBSYSTEM=="block",ATTR{partition}=="1",ATTR{size}=="1953521664",ATTRS{model}=="MQ01ABD100",ATTRS{vendor}=="TOSHIBA",
  2. SYMLINK+="drive1",
  3. RUN+="/bin/mount -o defaults,user,noexec /dev/drive1 /srv/nfs/drive1"

Part 1 is a list of attributes of the drive, which when taken together uniquely identify this drive. Udev rules can be looser than this (e.g. written to be triggered by all USB drives) but since the purpose of this rule is to mount a specific drive in a specific location, we don't want it to be triggered by any other drives (or udev would attempt to mount the other drives in the same location).

Part 2 is an instruction to create another device node in /dev/ for this drive (just for convenience). This symlink will create /dev/drive1. It's a symlink because the kernel name (/dev/sdX) will still exist, this node will be created using a symlink to the less useful name given by the kernel.

Part 3 is a script to be run when the device is inserted. As you can see, this command mounts the device at /srv/nfs/drive1 making use of the device node (/dev/drive1) created by the second part. Udev runs these commands in a weird environment without many of the environment variables you may expect to exist like $PATH. Therefore, you must use full paths to commands you specify in the RUN section.

So, how did I decide what to put in that first part? You can print information for a device using this command:

sudo udevadm info --attribute-walk /dev/sdXX

As stated in the output of the command:

"Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device."

The bold parts are important - you can't match using attributes of more than one parent devices in the same rule. The output for the relevant partition of my drive (/dev/sdb1) was as follows. I've made the parts used in my rule bold:

  looking at device '/devices/pci0000:00/0000:00:14.0/usb1/1-1/1-1:1.0/host2/target2:0:0/2:0:0:0/block/sdb/sdb1':
    KERNEL=="sdb1"
    SUBSYSTEM=="block"
    DRIVER==""
    ATTR{alignment_offset}=="0"
    ATTR{discard_alignment}=="0"
    ATTR{inflight}=="       0        0"
    ATTR{partition}=="1"
    ATTR{ro}=="0"
    ATTR{size}=="1953521664"
    ATTR{start}=="2048"
    ATTR{stat}=="     350       70    10314     2020       35       79      912      312        0     1628     2332"

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-1/1-1:1.0/host2/target2:0:0/2:0:0:0/block/sdb':
    KERNELS=="sdb"
    SUBSYSTEMS=="block"
    DRIVERS==""
    ATTRS{alignment_offset}=="0"
    ATTRS{capability}=="50"
    ATTRS{discard_alignment}=="0"
    ATTRS{events}==""
    ATTRS{events_async}==""
    ATTRS{events_poll_msecs}=="-1"
    ATTRS{ext_range}=="256"
    ATTRS{inflight}=="       0        0"
    ATTRS{range}=="16"
    ATTRS{removable}=="0"
    ATTRS{ro}=="0"
    ATTRS{size}=="1953525164"
    ATTRS{stat}=="     456       70    14682     3424       35       79      912      312        0     2484     3736"

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-1/1-1:1.0/host2/target2:0:0/2:0:0:0':
    KERNELS=="2:0:0:0"
    SUBSYSTEMS=="scsi"
    DRIVERS=="sd"
    ATTRS{device_blocked}=="0"
    ATTRS{device_busy}=="0"
    ATTRS{dh_state}=="detached"
    ATTRS{eh_timeout}=="10"
    ATTRS{evt_capacity_change_reported}=="0"
    ATTRS{evt_inquiry_change_reported}=="0"
    ATTRS{evt_lun_change_reported}=="0"
    ATTRS{evt_media_change}=="0"
    ATTRS{evt_mode_parameter_change_reported}=="0"
    ATTRS{evt_soft_threshold_reached}=="0"
    ATTRS{inquiry}==""
    ATTRS{iocounterbits}=="32"
    ATTRS{iodone_cnt}=="0x235"
    ATTRS{ioerr_cnt}=="0x0"
    ATTRS{iorequest_cnt}=="0x235"
    ATTRS{max_sectors}=="240"
    ATTRS{model}=="MQ01ABD100      "
    ATTRS{queue_depth}=="1"
    ATTRS{queue_type}=="none"
    ATTRS{rev}=="0016"
    ATTRS{scsi_level}=="5"
    ATTRS{state}=="running"
    ATTRS{timeout}=="30"
    ATTRS{type}=="0"
    ATTRS{vendor}=="TOSHIBA "

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-1/1-1:1.0/host2/target2:0:0':
    KERNELS=="target2:0:0"
    SUBSYSTEMS=="scsi"
    DRIVERS==""

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-1/1-1:1.0/host2':
    KERNELS=="host2"
    SUBSYSTEMS=="scsi"
    DRIVERS==""

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-1/1-1:1.0':
    KERNELS=="1-1:1.0"
    SUBSYSTEMS=="usb"
    DRIVERS=="usb-storage"
    ATTRS{authorized}=="1"
    ATTRS{bAlternateSetting}==" 0"
    ATTRS{bInterfaceClass}=="08"
    ATTRS{bInterfaceNumber}=="00"
    ATTRS{bInterfaceProtocol}=="50"
    ATTRS{bInterfaceSubClass}=="06"
    ATTRS{bNumEndpoints}=="02"
    ATTRS{supports_autosuspend}=="1"

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1/1-1':
    KERNELS=="1-1"
    SUBSYSTEMS=="usb"
    DRIVERS=="usb"
    ATTRS{authorized}=="1"
    ATTRS{avoid_reset_quirk}=="0"
    ATTRS{bConfigurationValue}=="1"
    ATTRS{bDeviceClass}=="00"
    ATTRS{bDeviceProtocol}=="00"
    ATTRS{bDeviceSubClass}=="00"
    ATTRS{bMaxPacketSize0}=="64"
    ATTRS{bMaxPower}=="96mA"
    ATTRS{bNumConfigurations}=="1"
    ATTRS{bNumInterfaces}==" 1"
    ATTRS{bcdDevice}=="0016"
    ATTRS{bmAttributes}=="c0"
    ATTRS{busnum}=="1"
    ATTRS{configuration}==""
    ATTRS{devnum}=="2"
    ATTRS{devpath}=="1"
    ATTRS{idProduct}=="0718"
    ATTRS{idVendor}=="05e3"
    ATTRS{ltm_capable}=="no"
    ATTRS{maxchild}=="0"
    ATTRS{product}=="USB Storage"
    ATTRS{quirks}=="0x0"
    ATTRS{removable}=="removable"
    ATTRS{serial}=="000000000033"
    ATTRS{speed}=="480"
    ATTRS{urbnum}=="1676"
    ATTRS{version}==" 2.00"

  looking at parent device '/devices/pci0000:00/0000:00:14.0/usb1':
    KERNELS=="usb1"
    SUBSYSTEMS=="usb"
    DRIVERS=="usb"
    ATTRS{authorized}=="1"
    ATTRS{authorized_default}=="1"
    ATTRS{avoid_reset_quirk}=="0"
    ATTRS{bConfigurationValue}=="1"
    ATTRS{bDeviceClass}=="09"
    ATTRS{bDeviceProtocol}=="01"
    ATTRS{bDeviceSubClass}=="00"
    ATTRS{bMaxPacketSize0}=="64"
    ATTRS{bMaxPower}=="0mA"
    ATTRS{bNumConfigurations}=="1"
    ATTRS{bNumInterfaces}==" 1"
    ATTRS{bcdDevice}=="0404"
    ATTRS{bmAttributes}=="e0"
    ATTRS{busnum}=="1"
    ATTRS{configuration}==""
    ATTRS{devnum}=="1"
    ATTRS{devpath}=="0"
    ATTRS{idProduct}=="0002"
    ATTRS{idVendor}=="1d6b"
    ATTRS{interface_authorized_default}=="1"
    ATTRS{ltm_capable}=="no"
    ATTRS{manufacturer}=="Linux 4.4.0-93-generic xhci-hcd"
    ATTRS{maxchild}=="6"
    ATTRS{product}=="xHCI Host Controller"
    ATTRS{quirks}=="0x0"
    ATTRS{removable}=="unknown"
    ATTRS{serial}=="0000:00:14.0"
    ATTRS{speed}=="480"
    ATTRS{urbnum}=="61"
    ATTRS{version}==" 2.00"

  looking at parent device '/devices/pci0000:00/0000:00:14.0':
    KERNELS=="0000:00:14.0"
    SUBSYSTEMS=="pci"
    DRIVERS=="xhci_hcd"
    ATTRS{broken_parity_status}=="0"
    ATTRS{class}=="0x0c0330"
    ATTRS{consistent_dma_mask_bits}=="64"
    ATTRS{d3cold_allowed}=="1"
    ATTRS{device}=="0x0f35"
    ATTRS{dma_mask_bits}=="64"
    ATTRS{driver_override}=="(null)"
    ATTRS{enable}=="1"
    ATTRS{irq}=="87"
    ATTRS{local_cpulist}=="0-1"
    ATTRS{local_cpus}=="3"
    ATTRS{msi_bus}=="1"
    ATTRS{numa_node}=="-1"
    ATTRS{subsystem_device}=="0x2055"
    ATTRS{subsystem_vendor}=="0x8086"
    ATTRS{vendor}=="0x8086"

  looking at parent device '/devices/pci0000:00':
    KERNELS=="pci0000:00"
    SUBSYSTEMS==""
    DRIVERS==""

You may find that you don't have to be as specific as I was (fewer attributes will probably be sufficient to uniquely identify a drive).

As mentioned before, add your rule to /etc/udev/rules.d/99-triggerDiskMountsForNFS.rules (remember to change the SYMLINK and the mount command!).

Udev should automatically detect changes to rules, so there should be no need to reload udev.

Test your rules by unplugging your external drive and then plugging it back in - you should find that it is mounted automatically in the right place.

Further notes/examples:

Note that the KERNEL/SUBSYSTEM/DRIVER/ATTR keywords only match root device, whereas KERNELS/SUBSYSTEMS/DRIVERS/ATTRS match against the device or any of the parent devices.

If you use the size of a partition as one of the identifiers, be sure to use the size of the partition and not the size of the whole drive!

Here are some further examples of udev rules for my drives. They are all pretty similar because in all cases I'm looking to mount partition 1:

SUBSYSTEM=="block",ATTR{partition}=="1",ATTR{size}=="976771072",ATTRS{model}=="External HDD",ATTRS{vendor}=="BUFFALO",SYMLINK+="drive2",RUN+="/bin/mount -o defaults,user,noexec /dev/drive2 /srv/nfs/drive2"
SUBSYSTEM=="block",ATTR{partition}=="1",ATTR{size}=="976752625",ATTRS{model}=="M3 Portable",ATTRS{vendor}=="Samsung",SYMLINK+="drive3",RUN+="/bin/mount -o defaults,user,noexec /dev/drive3 /srv/nfs/drive3"

If you need any further information about writing udev rules, see this excellent article. The only thing I couldn't find in that article is how to list attributes on Ubuntu, since the udevinfo command has been replaced by udevadm info, with usage as described in the example above.

If you're having problems, let me know in the comment section below, if not then continue to the next part of the series.

Type: 

Comments

Why not use fstab? here is how I do it: Barracuda and Caviar are two external drive in usb enclosures my fstab reads:

Get the filesystem uuids from blkid (not the partuuids...)

# Entry for caviar EXT4 :
UUID=71ca5d6d-9231-44fe-b50b-d525e68263ef /mnt/CaviarEXT4 ext4 defaults 0 0

# Entry for baracuda EXT4 :
UUID=18ba4f7c-ebec-40d1-9580-ff162fe262b1 /mnt/BaracudaEXT4 ext4 defaults 0 0

# Entry for baracuda NTFS :
UUID=8474A76C74A75FA2 /mnt/BaracudaNTFS ntfs-3g defaults,windows_names,nls=utf8,noexec,umask=000 0 0

If you fear the devices may be missing at boot, add the nofail option... I suppose you could also try the nowaitboot option although I am not sure this one is still supported...

and the nfs mounts:

#mounts for nfs
/mnt/BaracudaEXT4/MusicSource /srv/nfs4/musicsource none bind 0 0
/mnt/BaracudaNTFS/PourKodi /srv/nfs4/PourKodi none bind 0 0

/mnt/CaviarEXT4/VideoSource /srv/nfs4/videosource none bind 0 0

BTW, although the mount point says nfs4, make sure nfs 3 is available otherwise kodi won't like it...

Cheers, d.

Add new comment