how to persist ethtool settings across reboot

You can use the ethtool command to read and change information about your network interfaces.

For example, ethtool -g ethX reads the size of the ring buffer on the NIC, and ethtool -G ethx rx A tx B changes it. Use man ethtool to discover more settings.

But these options don't persist across reboot, so how do you make sure your settings are kept permanent?

You can enter the ethtool commands in /etc/rc.local (or your distribution's equivalent) where commands are run after the current runlevel completes, but this isn't ideal. Network services may have started during the runlevel and ethtool commands tend to interrupt network traffic. It would be more preferable to have the commands applied as the interface is brought up.

The network service in CentOS has the ability to do this. The script /etc/sysconfig/network-scripts/ifup-post checks for the existence of /sbin/ifup-local, and if it exists, runs it with the interface name as a parameter (eg: /sbin/ifup-local eth0)

We can create this file with touch /sbin/ifup-local make it executable with chmod +x /sbin/ifup-local set its SELinux context with chcon --reference /sbin/ifup /sbin/ifup-local and then open it in an editor.

A simple script to apply the same settings to all interfaces would be something like:

#!/bin/bash
if [ -n "$1" ]; then
    /sbin/ethtool -G $1 rx 4096 tx 4096
    /sbin/ethtool -K $1 tso on gso on
fi

Keep in mind this will attempt to apply settings to ALL interfaces, even the loopback.

If we have different interfaces we want to apply different settings to, or want to skip the loopback, we can make a case statement:

#!/bin/bash
case "$1" in
 eth0)
  /sbin/ethtool -G $1 rx 16384 tx 16384
  /sbin/ethtool -K $1 gso on gro on
  ;;
 eth1)
  /sbin/ethtool -G $1 rx 64 tx 64
  /sbin/ethtool -K $1 tso on gso on
  /sbin/ip link set $1 txqueuelen 0
  ;;
esac
exit 0

Now ethtool settings are applied to interfaces as they start, all potential interruptions to network communication are done as the interface is brought up, and your server can continue to boot with full network capabilities.

kvm virtual machine won't start

Recently I brought my KVM host down for an upgrade. I shut down the guests within their OSes, confirmed they were powered off with a virsh list --all, and everything went well with the upgrade.

However, upon returning the guests to service, one of them would not start up. Typing virsh start I got the helpful error message

Error restoring domain: cannot send monitor command
Connection reset by peer

Why was this one VM broken but the others were fine?

I played around with virsh autostart and virsh autostart --disable but that had no effect, nor did a reboot of the hypervisor.

After some searching around, it turns out libvirt has the capability to keep "managed save states" of guests, kinda like a sleep mode or snapshot, to save you fully powering a guest OS off.

For some reason, a managed save for this one guest had been created, perhaps it had not shut down properly, or perhaps there's an errorr in libvirt. I could view the saved state with

# virsh list --all --managed-save
 Id Name                 State
----------------------------------
  - guest1               shut off
  - guest2               saved

Now a virsh managedsave-remove guest2 returned it to the "shut off" state, and I could start it properly with virsh start as per usual.