Fix for Ethernet Connection Drop on Raspberry Pi

Raspberry_Pi_Ethernet_Port.jpg

The Problem

Today an engineer from BT was fiddling with the junction box outside my house, and my modem dropped connection to my router.

At the time, the router did not automatically force reconnect (my fault, I hadn’t configured it to do so). When I noticed what had happened, I reconnected to the modem. So far so good.

A couple of hours later, I noticed that two of my three Pi (all of which are connected with ethernet cables) had not reconnected to the router. The one that did reconnect is running Raspbmc (XBMC port to Raspberry Pi); the two that did not are running Apache with some bits on top (a mail server, owncloud, and wordpress for this website!).

This is a pain because not only did it take the services offline, but I was unable to SSH to the Pi to correct the problem. Removing and reconnecting the ethernet cables did not work, so in the end I had to pull the power and reboot.

Existing Partial Solutions

I found a thread on the RasPi forums about reconnecting WiFi with a BASH script after a drop. My guess is that Raspbmc includes something similar, but for ethernet, which is why it reconnects and the other two Pi do not:

#!/bin/bash

while true ; do
   if ifconfig wlan0 | grep -q "inet addr:" ; then
      sleep 60
   else
      echo "Network connection down! Attempting reconnection."
      ifup --force wlan0
      sleep 10
   fi
done

The directions on the thread indicated that this script could be run in the background (i.e. sudo network-manager.sh &), and added to the end of /etc/rc.local so that it runs when the system is first booted.

Clearly, this solution needed modification to work for ethernet instead of wifi, but I was also concerned that running a script all the time in the background could be an unnecessary drain on the Pi’s resources. One more thing that I wanted the script to do was tell me what had happened during the time that the Pi was reconnected. These factors led me to create a new script for my particular problem.

My Solution: New BASH Script

This section presents and my script, and how to use it.

Saving a copy of the script

If you don’t already have a subfolder inside your user’s home folder for scripts, create one now:

mkdir ~/bin

Now copy this script into your favourite text editor, and save it as network-monitor.sh inside that folder.

#!/bin/bash

LOGFILE=/home/admin/network-monitor.log

if ifconfig eth0 | grep -q "inet addr:" ;
then
        echo "$(date "+%m %d %Y %T") : Ethernet OK" >> $LOGFILE
else
        echo "$(date "+%m %d %Y %T") : Ethernet connection down! Attempting reconnection." >> $LOGFILE
        ifup --force eth0
        OUT=$? #save exit status of last command to decide what to do next
        if [ $OUT -eq 0 ] ; then
                STATE=$(ifconfig eth0 | grep "inet addr:")
                echo "$(date "+%m %d %Y %T") : Network connection reset. Current state is" $STATE >> $LOGFILE
        else
                echo "$(date "+%m %d %Y %T") : Failed to reset ethernet connection" >> $LOGFILE
        fi
fi

i.e.

nano ~/bin/network-monitor.sh

…then copy & paste, save and exit (Ctrl + X, hit yes when prompted to save). Make sure the long lines don’t get truncated when you copy and paste them over, or the script won’t work!

Note that the script points to a file that will be used as a log on line 3. My username is “admin”; if yours is “pi” then change that line to LOGFILE=/home/pi/network-monitor.log, or replace “pi” with any other username.

Adding ~/bin to your $PATH variable, and making the script executable

Your $PATH variable is a list of places that the shell looks for executables. We need to add the newly added ~/bin so that you don't have to use the full path to the script when you run it. To do this, open ~/.bashrc and add this line to the end:

PATH=$PATH:~/bin

The shell normally reads ~/.bashrc when you log in, but we can tell it to parse it now using this command to that the changes take effect:

source ~/.bashrc

Now we need to make the script executable, e.g.:

chmod +x ~/bin/network-monitor.sh

Initial test

You should now be able to run the script using sudo (i.e. sudo network-monitor.sh). Check the log file with this command:

less ~/network-monitor.log

You should see a date and time stamp from when you ran the script, followed by “Ethernet OK”. If this is what you see, all is well. Press q to quit “less” and return to the command prompt when you are done.

Automating with cron

Cron is a tool that can run scripts at regular intervals, and is very suited for this kind of thing. Luckily, it’s really easy to get started with.

Open the cron configuration file with root privileges:

sudo nano /etc/crontab

Now schedule the script to be run every 5 minutes (or any other interval that you would prefer), by adding this last line to the end of the file (remember to change “admin” if you have a different username):

# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user  command
17 *    * * *   root    cd / && run-parts --report /etc/cron.hourly
25 6    * * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.d$
47 6    * * 7   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.w$
52 6    1 * *   root    test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.m$
#

*/5 * * * * root bash /home/admin/bin/network-monitor.sh

If you want the script to run every 10 minutes (or any other multiple of 1 minute), change the first part of the line, e.g. to “*/10″. It’s important to set the script to run with root because the ifup command requires superuser privileges.

Make a cup of tea and come back. Check the log file again, you should see more entries with time stamps that are 5 minutes (or whatever you specified) apart.

Full Test

Now is the time to test your script to see if it will reconnect you when the ethernet connection goes down.

I’d recommend that you have physical access to the Pi when you test this for the first time, just in case you have to pull the power and reboot (very unlikely if everything has worked so far).

Make sure that your Cron interval isn’t set too high or you’ll have to wait ages (5 minutes is about right for testing).

Connect to the Pi via SSH, and issue this command (after you issue it, your SSH session will become unresponsive, but that’s kind of the point!):

sudo ifdown eth0

Wait for at least the amount of time you specified between cron jobs, and then login again. You may actually still be logged in – if you didn’t try and get the Pi to do anything while the ethernet was down, it may not have registered the connection drop.

Check the log file. You should see something like this:

11 10 2013 16:50:01 : Ethernet OK
11 10 2013 16:55:02 : Ethernet OK
11 10 2013 17:00:01 : Ethernet OK
11 10 2013 17:05:01 : Ethernet connection down! Attempting reconnection.
11 10 2013 17:05:01 : Network connection reset. Current state is inet addr:192.168.1.103 Bcast:192.168.1.255 Mask:255.255.255.0
11 10 2013 17:10:01 : Ethernet OK

Success! Your Pi can now successfully reconnect to the router. If the router itself was down for a period of time, then you would see something like this in the logs, with the script attempting to reconnect and failing until the router was back up:

11 10 2013 16:50:01 : Ethernet OK
11 10 2013 16:55:02 : Ethernet OK
11 10 2013 17:00:01 : Ethernet OK
11 10 2013 17:05:01 : Ethernet connection down! Attempting reconnection.
11 10 2013 17:05:01 : Failed to reset ethernet connection
11 10 2013 17:10:01 : Ethernet connection down! Attempting reconnection.
11 10 2013 17:10:01 : Failed to reset ethernet connection
11 10 2013 17:15:01 : Ethernet connection down! Attempting reconnection.
11 10 2013 17:15:01 : Failed to reset ethernet connection
11 10 2013 17:20:01 : Ethernet connection down! Attempting reconnection.
11 10 2013 17:20:01 : Network connection reset. Current state is inet addr:192.168.1.103 Bcast:192.168.1.255 Mask:255.255.255.0
11 10 2013 17:25:01 : Ethernet OK

Future Improvements

I may do some further work on the script to combine all “Ethernet OK” lines into a single line containing two time stamps, between which the connection was fine, e.g:

11 10 2013 16:35:02 to 11 10 2013 17:00:01 : Ethernet OK
11 10 2013 17:05:01 : Ethernet connection down! Attempting reconnection.
11 10 2013 17:05:01 : Network connection reset. Current state is inet addr:192.168.1.103 Bcast:192.168.1.255 Mask:255.255.255.0

For now, I’m just glad that it works!

If you think you can improve the script, please let me know, I’d love to hear your suggestions!

Category: 
Type: 

Comments

Hello,

What you wrote is exactly what I want. You can share your script please?
PS: And the "job to run every hour to clear the Logfile". :)
Thanks

Hello asm19:
I'd be more than happy to try to help if you can bear with me. Since using this script, modified, I haven't had any need to go back into my Pi. It just runs. So it may take me a few minutes, when I'm home, to go back in and figure out what I did....no a biggie though. On top of that I've recently contracted Lyme disease so everything, at least for the time being, I do I do at a snails pace....totally annoying.
So if you aren't strapped for time I will get to it and post it. I thought, but I can't seem to find it, that I had posted what I had done, but maybe it wasn't clear and/or intuitive enough for someone else to follow...my apologies.
I will say though that I searched a good deal before arriving at this page and the base script here is the way to go.

asm19: Below is how I modified the script for my use. It's been running since last August without a single issue.

#!/bin/bash

LOGFILE=/home/pi/network-monitor.log

if ifconfig eth0 | grep -q "inet addr:" ;
then
if ping -c 1 10.8.0.1 &> /dev/null
then
echo "$(date "+%m %d %Y %T") : Ethernet and VPN OK" >> $LOGFILE
else
echo "$(date "+%m %d %Y %T") : Ethernet OK - VPN down" >> $LOGFILE
sudo /etc/init.d/openvpn restart
echo "$(date "+%m %d %Y %T") : Restarting VPN." >> $LOGFILE
fi
else
echo "$(date "+%m %d %Y %T") : Ethernet connection down! Attempting reconnection." >> $LOGFILE
echo "$(date "+%m %d %Y %T") : Sleeping for 3min." >> $LOGFILE
sleep 3m
echo "$(date "+%m %d %Y %T") : Resuming program." >> $LOGFILE
ifup --force eth0
OUT=$? #save exit status of last command to decide what to do next
if [ $OUT -eq 0 ] ; then
STATE=$(ifconfig eth0 | grep "inet addr:")
echo "$(date "+%m %d %Y %T") : Network connection reset. Current state is" $STATE >> $LOGFILE
echo "$(date "+%m %d %Y %T") : Restarting VPN." >> $LOGFILE
sudo /etc/init.d/openvpn restart
else
echo "$(date "+%m %d %Y %T") : Failed to reset ethernet connection" >> $LOGFILE
fi
fi

Hi Sam,

my Pi2 had this ethernet-loss-problem. And since this Pi is my NAS which has my media files, it bothered me a lot.
Thanks to your network-monitor.sh the problem is solved. Works absolutely fine.
All I had to do is change "inet addr" to its German translation.
So thanks again for sharing.

Hi Sam,

Thanks for the script! I'd changed my /etc/network/interfaces file to give my RPi a static IP (iface eth0 inet static) but when one issues ifup en0 this is ignored and the device gets a standard DHCP assignment. Is there a way round that?

Cheers

Sam,
I installed your script on my raspberry pi running wview. No problems restarting the link when i run the ifdown eth0 command. When I disconnect the router, however, can't get the ethernet link re-established. I'm a newbie to all this but any suggestions gratefully received.

Steve

Sam,
I ran a couple of different tests to see if I could help locate the problem.
1. Shut the ethernet link with the "ifdown eth0" command. The link reset but log information not the same as you quote in your post. Full ip details not published.
Shutdown using ifdown eth0 command. Reset successful
09 16 2015 18:50:01 : Ethernet OK
09 16 2015 18:55:01 : Ethernet connection down! Attempting reconnection.
09 16 2015 18:55:03 : Network connection reset. Current state is inet addr:192.$
09 16 2015 19:00:01 : Ethernet OK
2. Shut down and started modem between 5 min checks. Script didn't seem to recognise link was down but definitely couldn't connect using putty
Turned off 1822, on at 1823. Didn’t reset
09 16 2015 18:15:01 : Ethernet OK
09 16 2015 18:20:02 : Ethernet OK
09 16 2015 18:25:01 : Ethernet OK
09 16 2015 18:30:01 : Ethernet OK
3. Tried turning modem off just before 5 min check and turning on just after 5 min check. Log file suggested the link had been reset but link with putty still definitely down
Turned off 1839, on at 1841. Didn’t reset
09 16 2015 18:35:01 : Ethernet OK
09 16 2015 18:40:02 : Ethernet connection down! Attempting reconnection.
09 16 2015 18:41:20 : Network connection reset. Current state is inet addr:192.$
09 16 2015 18:45:01 : Ethernet OK
09 16 2015 18:50:01 : Ethernet OK
4. Tried turning modem off for several 5 min checks. Script suggested link had been reset but still couldn't connect with putty
Modem shutdown 1905, on at 1925. Didn’t reset
09 16 2015 19:00:01 : Ethernet OK
09 16 2015 19:05:01 : Ethernet connection down! Attempting reconnection.
09 16 2015 19:05:03 : Network connection reset. Current state is inet addr:192.$
09 16 2015 19:10:01 : Ethernet connection down! Attempting reconnection.
09 16 2015 19:11:19 : Network connection reset. Current state is
09 16 2015 19:15:01 : Ethernet connection down! Attempting reconnection.
09 16 2015 19:16:19 : Network connection reset. Current state is
09 16 2015 19:20:01 : Ethernet connection down! Attempting reconnection.
09 16 2015 19:21:19 : Network connection reset. Current state is
09 16 2015 19:25:02 : Ethernet connection down! Attempting reconnection.
09 16 2015 19:25:49 : Network connection reset. Current state is inet addr:192.$
09 16 2015 19:30:01 : Ethernet OK
09 16 2015 19:35:01 : Ethernet OK
09 16 2015 19:40:01 : Ethernet OK

Hi,
I'm using your script. But I have one doubt. I'm using static ip address. If my raspbian loose connection, and your script needs to activate, what happens? Give the static ip address that was using?

Hi,

It will bring the interface back up according to whatever settings you have configured (e.g. in /etc/network/interfaces).

Try taking it down manually and see if it reconnects.

Sam

Hi!

Done!!! 5 stars!!!!
one more thing, if I set on rc.local like this
/var/script/network-monitor.sh do I need to set under crontab? On my opinion, if I use rc.local, I don't need to use it on crontab, cause on rc.local stays allways on background. Am I right?

Never used rc.local myself but from what I understand it's run once at boot and that's it, so that wouldn't be good enough. The script is called from scratch every 5minutes or whatever, it's not a long running process/service (although you could make it one by putting a sleep loop in).

Sam

I had the same issue that you had after my connection from my ISP to my router went down.

I couldn't figure out why your solution would not work for me, but a reboot did. I was automatically connecting to my VPN service at boot, so after 'sudo ifup --force eth0', I added the command: 'sudo /etc/init.d/openvpn restart' and that did the trick!

Hopefully this may help someone else who stumbles upon this same issue.

When i do "sudo ifdown eth0" i get error "eth0 not vonfigured". But i am using eth0(default) even when i hover mouse on internet icon on right aide of taskbar i can see "eth0:configured:192.168.2.2/24"
Please help me.

Hi,

ifup and ifdown only work if your network is configured in /etc/network/interfaces.

If you're running a graphical desktop that might not be the case (but then networkmanager or something similar might be handling reconnection anyway, so you might not need the script at all.

Sam

i am running graphical desktop but i do get occasional network disconnection problem and my ssh especially my torrents downloading stops but my display on my monitor still runs good everyting runs ok no freezing or anything just network problem. i used ur script but have not yet seen any network related problem in log file yet.
please give me ur precious reply when i get the network problem next time will the script work ?

I don't think it will work if you're using network management, but I'm not certain.

The problem you are having is probably due to the fact that the hardware design of the RasPi means the USB ports and ethernet share a USB hub - when you start to get large amounts of traffic through the ethernet port and the USB ports at the same time, you're likely to start dropping packets. Torrenting obviously meets this description, so that's probably why you've had problems.

Sam

Hi,
I am grateful for your script, but somehow my copypasted version does not work. It keeps logging Ethernet OK, even after I tested it with sudo ifdown eth0. I lost my connection, but I had to reset my pi manually. I donw know how to fix this. Wpuld be grateful for help!

Hi John,

To investigate, I would connect a monitor and keyboard to the pi, log in locally and kill the connection with ifdown, then see what you get from ifconfig eth0, that might tell us why it's not working.

Sam

Thank you for your feedback, turns out I had two ips configured in my settings. The script worked perfectly, just connected to different ip. All fixed now, Thank you again.

For some reason on the version of Raspbian I had freshly installed on the Rpi 2, ifup eth0 and ifdown eth0 didn't recognise eth0 at all. Simply changing the command to ifconfig eth0 up or ifconfig eth0 down worked a treat.

Thanks very much for this! It's a shame the Pi doesn't try to reconnect by itself.

That's interesting, thanks.

I think it will reconnect by itself if you use NetworkManager or Wicd, it's just the default static configuration that doesn't.

Sam

Great work Sam, and nicely demonstrated. I'm using it on my headless Pi Zero W when I suddenly discovered my "no-ip" DDNS script wouldn't re-connect following a router restart.... Thanks again - saved a bit of messing-about to end up with something not quite as good :)

Hi,

thanks for this script. I tried it, but figured out some problems.

I saved it, made it executable.

When i execute it manually, it works perfectly.
When i start it via cron, it is executed, but, it always says network down,... but it is up.

06 24 2017 11:45:02 : Ethernet connection down! Attempting reconnection.
06 24 2017 11:45:02 : Failed to reset ethernet connection
06 24 2017 11:46:01 : Ethernet connection down! Attempting reconnection.
06 24 2017 11:46:01 : Failed to reset ethernet connection
06 24 2017 11:47:02 : Ethernet connection down! Attempting reconnection.
06 24 2017 11:47:02 : Failed to reset ethernet connection

Any Ideas?

Thanks for your help

Joerg

Pages

Add new comment