Configure Open vSwitch on Scientific Linux 6 and 7

Configuring OVS for Testing and Use

Now that we have installed Open vSwitch on our system, we need to insert it into our network configuration. The plan is to create OVS bridges and insert them into the network between the machine's external interfaces and the Linux network stack. Changing the configuration of a production network can cause service interruptions and needs to be done with care. We will describe a system that allows for fast recovery if OVS insertion for some reason breaks connectivity. Most of the action here revolves around the network configuration files in the "/etc/sysconfig/network-scripts/" directory, which are any files in that directory beginning with "ifcfg-" or "route-" and ending with an interface name.

We assume in the following instructions that the interfaces you want to reconfigure have a static IP address. We also recommend that you disable Network Manager if doing so would not disrupt your configuration.

Setting up to Recover

We have prepared a shell script, prep_ovs.sh (see attachments) that will do all the preparation described below, setting up ONLY THE PUBLIC NIC for Open vSwitch use. This script creates the described "network-scripts" sub-directories if they do not already exist (and errs out if they do exist), and fills them with all needed content. Note that this script has one, configurable line, that must be edited to define the public IP subnet string that will be used. This allows the script to find and modify the correct ifcfg files. For AGLT2, this line is

• public_IP_prefix="192.41"

prep_ovs.sh

One further expectation is that all of the scripts, or at least the "clean-nc.sh" script, should be placed into a directory called "/root/tools".

Recovery Setup Details

In order to recover a past network configuration quickly we will save the current ("old") configuration files to a sub-directory, create a few scripts to move the new or old set of configuration files into the /network-scripts/ directory, and outline the necessary command line tools.

To begin, create two new directories: /etc/sysconfig/network-scripts/old_config and /etc/sysconfig/network-scripts/new_config. Then copy existing configuration files into the old_config directory, as root:

cd /etc/sysconfig/network-scripts

mkdir old_config new_config

cp ifcfg-* route-* old_config/

Now we have a saved copy of the current network configuration. We will also need a few scripts (to be saved to the 'network-scripts' directory)

clean-nc.sh Deletes all configuration files in the base 'network-scripts' directory, to be used as a part of later scripts:

del-all-br.sh Deletes all OVS bridges - this can be useful in quickly reverting to the old configuration (assuming no OVS bridges were present in that configuration). If necessary, individual bridges can be deleted by using the OVS command

ovs-vsctl del-br <BRIDGE_NAME>

new_config.sh Deletes all configuration files in the 'network-scripts' directory and then puts the new configuration files in place from the 'new_config' subdirectory

old_config.sh Deletes all configuration files in the 'network-scripts' directory and then restores the old configuration files from the 'old_config' subdirectory

You can copy these scripts (attached to this wiki page) to /root/tools/ and make them executable with

chmod a+x clean-nc.sh del-all-br.sh old_config.sh new_config.sh prep_ovs.sh

At this point we can edit ifcfg-* files to define a new network configuration that inserts an OVS bridge on each selected interface. The ifcfg-* files define interface and bridge parameters that take effect when the network service is started - changing the files in 'network-scripts' will have no effect on the state of the network service until the network service, or the individual interface, is restarted. Nevertheless, we suggest copying the old configuration into the 'new_config' directory, doing your editing there, and then copying the new configuration into the network_scripts directory using the new_config.sh script.

In order to recover quickly we may also need to delete any virtual bridges we've created. If you simply revert to the old set of network configuration files the routes associated with the ovs bridge might not be changed, which can break connectivity.

Modifying Network Configuration Files

We can easily convert an existing interface configuration file (which defines an ethernet port, VLAN subport, or bonded port) into a pair of configuration files, one for the original port and another for an OVS bridge to insert between the port and the SL network stack. Note: We have not explored how route-* files should be handled.

So long as OVS is running, the configuration file which defines each bridge will cause network service startup to create the bridge, fully configured as defined in the file.

There are a number of possible configurations for adding an OVS bridge, but without a specific need for something more complex it seems easiest to create a separate bridge for each interface.

The following represents a sample Ethernet interface configuration file (named ifcfg-<iface_name>, in this case ifcfg-eth1) translated into the corresponding set of interface and bridge configuration files (ifcfg-eth1 and ifcfg-ovsbreth1, where the bridge name "ovsbreth1" can be whatever you decide). The IP information (in green) should be removed from the interface file and added to the bridge file. The lines in blue should be changed or added to the interface file; note that the bridge that the interface will be attached to needs to be specified in the interface file (shown in red).

Original Configuration:

ifcfg-eth1

DEVICE=eth1
ONBOOT=yes
HWADDR=<MAC address>
TYPE=Ethernet
BOOTPROTO=none
IPADDR=192.168.1.79
NETMASK=255.255.255.0
NM_CONTROLLED=no

New Configuration:

ifcfg-eth1

DEVICE=eth1
ONBOOT=yes
HWADDR=<MAC address>
BOOTPROTO=none
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=ovsbreth1
NM_CONTROLLED=no

ifcfg-ovsbreth1

DEVICE=ovsbreth1
ONBOOT=yes
DEVICETYPE=ovs
TYPE=OVSBridge
OVSBOOTPROTO=none
IPADDR=192.168.1.79
NETMASK=255.255.252.0
NM_CONTROLLED=no

Bonded and VLAN Interfaces

We have found it easiest to treat bonded and VLAN interfaces the same way we do individual Ethernet interfaces -- attach the VLAN interface or the bonded interface to a unique OVS bridge. The original interface configuration file will look a little different from the Etherent case; however, the edits are the same. The following examples show the new interface configuration files, using the convention of naming the OVS bridge by prepending "ovsbr" to the interface name. The OVS bridge configuration files will look the same as in the Ethernet example (except for the DEVICE name and corresponding file name) with the bridge taking the IP address and subnet mask from the original interface.

Bonded Interface

ifcfg-bond0

DEVICE=bond0
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
BONDING_OPTS="mode=1 miimon=100"
ONBOOT=yes
BOOTPROTO=none
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=ovsbrbond0
NM_CONTROLLED=no

VLAN

ifcfg-eth1.100

DEVICE=eth1.100
VLAN=yes
ONBOOT=yes
BOOTPROTO=none
DEVICETYPE=ovs
TYPE=OVSPort
OVS_BRIDGE=ovsbreth1_100
NM_CONTROLLED=no

Starting the new configuration

All that remains in order to insert OVS into the network configuration is to move the new configuration files into the 'network-scripts' directory and restart the network service. Although a network restart should in principle be sufficient after moving new ifcfg files into place, we have noted that in fact it works more cleanly to stop the network, move files, then start the network. Both methods are shown below. For the former, we note that services seem happier with a short sleep before the network is started again. The nets are down for anywhere from 20s to 1m.

in SL 6:

service network stop; ./new_config.sh; sleep 10; service network start

or

./new_config.sh; service network restart

in SL 7:

systemctl stop network; ./new_config.sh; sleep 10; systemctl start network

or

./new_config.sh; systemctl restart network

Be aware that, if there is an error in the configuration files, restarting the network service can break the machine's connection to the network. Before you restart the network service (or make any OVS bridges using the CLI) make sure that you have a plan to recover connectivity. If you are connected to the machine over the network, you may lose contact with the machine and be unable to take steps to recover. Ultimately the only safe way to run this is on the console of the host (or over a KVM to the console or over a Serial-Over-Lan (SOL) connection).

Reverting to the original configuration

We can create an OVS bridge using an appropriate configuration file, but we cannot delete a bridge by removing the configuration file. Removing the configuration file for a bridge and restarting the network service will make no change to the bridge - it will still have the same port associations and the same IP address, and may still be in the routing table, after the restart. If the new configuration does not work as intended you can certainly use the ovs command line to troubleshoot, but if you cannot tolerate downtime, the fastest and surest way to revert to the old configuration is to simply put the old configuration files in place, delete the relevant OVS bridge(s), and restart the network service. If you have no attachment to any of your OVS bridges, you can opt to delete them all:

in SL 6:

service network stop; ./old_config.sh; ./del-all-br.sh; sleep 10; service network start

or

./old_config.sh; ./del-all-br.sh; service network restart

Otherwise, you can delete only the relevant bridge using the OVS command:

service network stop; ./old_config.sh; ovs-vsctl del-br <bridge_name>; sleep 10; service network start

or

./old_config.sh; ovs-vsctl del-br <bridge_name>; service network restart

If you are reverting to an old configuration over the network, be aware that deleting the bridge and restarting the network service should be done in the same step.

in SL7:

systemctl stop network; ./old_config.sh; ./del-all-br.sh; sleep 10; systemctl start network

or

./old_config.sh; ./del-all-br.sh; systemctl restart network

or

systemctl stop network; ./old_config.sh; ovs-vsctl del-br <bridge_name>; sleep 10; systemctl start network

or

./old_config.sh; ovs-vsctl del-br <bridge_name>; systemctl restart network

-- GalenMackCrane - 11 May 2017