Broadcom Advanced Server Program (BASP) Driver Software (Linux): Broadcom NetXtreme® 57XX User Guide
NOTES:
- Broadcom supports the advanced BASP features only on Red Hat Enterprise Linux.
- For further information regarding teaming, see Teaming and Broadcom Gigabit Ethernet Teaming Services.
Installing Broadcom Advanced Server Program
Installing Broadcom NICE Patches
Uninstalling the BASP RPM Package
Removing a Physical Interface in Generic Trunking and IEEE 802.3ad Mode
Overview
Broadcom Advanced Server Program is a kernel module designed for Linux 2.4.x and 2.6.x kernels. See Broadcom Advanced Server Program Overview in Teaming for a detailed description.
Broadcom Advanced Server Program also provides remote management through the SNMP protocol, and this package is installed separately (see Installing BASP SNMP Agent).
Packaging
Broadcom Advanced Server Program is released in 2 packaging formats: source RPM and compressed tar formats. The file names for the 2 packages are basplnx-version.src.arch.rpm and basplnxversion.arch.tgz, respectively. Identical source files to build the driver are included in both RPM and TAR source packages.
Installing Broadcom Advanced Server Program
Broadcom Advanced Server Program for Linux is shipped in mixed forms; the platform and kernel-specific files are in source code, and the core file is in object form. Four packages are shipped in this release: three RPM packages and one tar archive. The tar archive for i386 platform is basplnx-version.i386.tgz.
- Red Hat distribution users. See Installing the BASP Source RPM Package.
- Linux i386 distribution users. See Installing the BASP TAR Archive.
Installing the BASP Source RPM Package
- Run the following:
% rpm -i basplnx-version.src.archrpm- Change to the directory containing the RPM package and build the binary driver for the kernel (use rpmbuild for Red Hat Linux 2.1 and 3.0 or later). The path to the RPM package is different for different Linux distributions.
% CD /usr/src/redhat% rpm -bb SPECS/basplnx.specor
rpmbuild -bb SPECS/basplnx.spec
NOTE: During the installation process, the following message may be displayed:
error: cannot create %sourcedir /usr/src/redhat/SOURCESThe most likely cause of the error is that the rpm-build package has not been installed. To install the rpm-build package, see Installing the BASP Source RPM Package.See Configuring Teaming for Linux Red Hat Distributions to set up the teams.
Installing the BASP TAR Archive
Uncompress and expand the tar archive:
% tar xvfz basplnx-version.arch.tgzTo install the BASP TAR archive
- Change to the directory where the BASP source files are located:
% CD basplnx-version- Build the kernel module, basp.o:
% make
NOTE: The Make process automatically builds the correct module for different kernel options, for example, symbol version and SMP support. There is no need to define -DMODVERSIONS in the Makefile.
- Create a device file and copy files:
% make install- Update the module reference:
% depmod -a- Load the driver:
% modprobe baspSee Configuring Teaming for Other Linux Distributions to set up the teams.
BASP Files
Configuring Teaming
Configuring Teaming for Linux Red Hat Distributions
NOTES:
- If you do not enable LiveLink™ when configuring teams, disabling Spanning Tree Protocol (STP) at the switch is recommended. This minimizes the downtime due to spanning tree loop determination when failing over. LiveLink mitigates such issues.
- When adding 64 VLANs, the 64th VLAN must have a VLAN ID of 0 (63 VLANs are tagged and 1 VLAN is untagged).
- Although teaming increases the load on the CPU, teaming does not change the CPU utilization rate.
- VLANs are not supported on non-Broadcom adapters. It is supported on the Alteon® adapters if the ALT.LAN driver provided by Broadcom is used. If a non-Broadcom adapter is a member of a failover team, VLANs are not supported for that team.
- If remote management (IPMI) is enabled on an adapter, the IPMI service could be disrupted when the adapter is added to a team other than an SLB team.
Because Red Hat distribution installations do not automatically load drivers for network devices unless the device is configured with an IP address, you must manually configure a network script file for all of the physical (installed) adapters that are potential team members. Network script files are located in the /etc/sysconfig/network-scripts file. The file name of the script file must be prefixed with ifcfg- and followed by the physical adapter alias. For interface eth0, for example, create a file with the name ifcfg-eth0,and then add the content, as shown in the following example.
Example:
DEVICE = eth0
BOOTPROTO = static
ONBOOT = yesBroadcom Advanced Server Program includes several scripts for configuring teams for Linux Red Hat distributions.
NOTE: Enabling Dynamic Host Configuration Protocol (DHCP) is not recommended for members of an SLB team
To configure a team
See Using BASP Scripts for Configuring Teams for the syntax of the configuration script or in the /etc/basp/sample/team-sample script file. When configuring teaming, you must designate at least one adapter as a primary team member.
- Manually start the team for the first time:
% /etc/init.d/basp start
NOTES:
- This step is required only for the initial installation; the team configuration is automatically started on subsequent reboots.
- If not all the virtual network interfaces are configured with an IP address, an error in starting the BASP team results. When this happens, modify the configuration script to assign an IP address for all of the virtual network interfaces.
- Forming multiple teams is possible by copying the sample files to the /etc/basp/team-name file and modifying this file as described in the sample file.
- For instructions on how to create more that one virtual interface (VLAN) for each team, refer to the appropriate section in the sample files.
Using BASP Scripts for Configuring Teams
The team-sample configuration script creates an SLB team named Team1. The team consists of 3 network interfaces, eth0, eth1, and eth2. All 3 interfaces are primary adapters. One virtual interface named sw0 is added to the team and the VLAN is not enabled. This script is part of the Broadcom Advanced Server Program driver for Linux distributions.
The syntax used in the sample scripts, team-sample and team-gec (see BASP Files) is the same as the syntax shown in the following table:
NOTE: Teaming scripts are intended for Red Hat distributions only. Do not use teaming scripts with other Linux distributions.
Configuring Teaming for Other Linux Distributions
Broadcom Advanced Server Program Configuration (baspcfg) is a command-line tool to configure the BASP teams, add/remove NICs, and add/remove virtual devices. This tool can be used in custom initialization scripts. Read your distribution-specific documentation for more information on startup procedures.
Example:
baspcfg v6.2.7 Broadcom Advanced Server Program Configuration Utility Copyright (c) 20002004 Broadcom Corporation. All rights reserved.Usage:
baspcfg command
BASP Configuration Commands
BASP Configuration Command Placeholders
NOTE: The baspcfg command can be run only in Super User mode. Attempting to run baspcfg as a standard user returns the following error message:
Error in communicating to BASP Module. Is it loaded?When configuring teaming, you must designate at least one adapter as a primary team member.
Configuring LiveLink
Read the following notes before you attempt to configure LiveLink™.
NOTES:
- Before you begin, review the description of LiveLink™ Functionality.
- A probe target must be on the same subnet as the team, have a valid (not a broadcast, multicast, or unicast), statically-assigned IP address, and be highly available (always on).
- You can specify up to 4 probe targets.
To configure LiveLink
- Open the BASP directory.
/etc/basp- Change to the SAMPLES directory.
cd samples- Copy the team-sample file to the BASP directory.
cp /etc/basp/samples/team-sample /etc/basp/- Edit the team-sample script to enable LiveLink, assign IP addresses to probe targets, assign IP addresses to team members, specify the time interval for sending link packets to probe targets, and to specify the maximum number of probe retries. You must delete the # sign at the beginning of each respective line item.
- Go to #LIVELINK_ENABLE=0 and change the value from 0 to 1.
- Go to #PROBE_TARGET_IP0= and type the target IP address.
NOTE: Only the first probe target is required. You can specify up to 3 additional probe targets to serve as backups by assigning IP addresses to the other #PROBE_TARGET_IPx= line items
- Go to #TEAM_PA0_IP= (the first physical Interface in the team) and type the team member IP address.
NOTE: You must assign an IP address to each team member for that member interface to be used by the team.
- Go to #PROBE_INTERVAL=5 and accept the default value (recommended) by deleting the # sign. To specify a different value, type the desired value in seconds within the predefined range.
- Go to #MAX_RETRY=5 and accept the default value (recommended) by deleting the # sign. To specify a different value, type the desired value in seconds within the predefined range.
- Save the file.
NOTES:
- Before you start the team, it is recommended that you use the ifconfig command to temporarily bring up one of the member interfaces using the IP address that you specified for that member in step 4.C. Then ping the probe target that you specified in step 4.B to verify connectivity. After you confirm that there is connectivity, type the ifconfig down command to bring down the interface. Now you are ready to start the team.
- To view the status of LiveLink activity (the number of probes sent and the number of probe retries) for the team, use the baspcfg show command.
CAUTION! If the IP address of the probe target is changed, the value of PROBE_TARGET_IPx must be updated, and the team must be restarted.
To configure LiveLink in VLAN-tagged environments
CAUTION! For the teams with VLANs (on which LiveLink is enabled): to be able to communicate with the probe target, both the probe target and the team must be on VLAN ID 0 (untagged). Otherwise, the team loses connectivity.
- Ensure that the team has an untagged VLAN (VLAN ID 0).
- Ensure there is network connectivity between the team and the probe target on the untagged VLAN.
- Configure LiveLink as specified in Configuring LiveLink.
Installing Broadcom NICE Patches
Also included in this release are network device drivers patched with Broadcom NICE support. These drivers are originally taken from the Linux 2.4.16 kernel distribution. To install patched drivers:
- Copy the Broadcom NICE header file (nicext.h) to the appropriate Linux kernel include directory:
% cp /usr/src/nice-2.4.16/nicext.h /usr/src/linux/include/linux- Rename the original network device driver under the Linux kernel source tree (/usr/src/linux/drivers/net).
- Copy the patched drivers to the Linux kernel network driver source directory (for example, /usr/src/linux/drivers/net).
- Follow the kernel rebuild instructions to configure kernel support for these drivers:
% CD /usr/src/linux% make config- If the patched drivers are configured into the kernel, go to step 7. If the patched drivers are configured as modules, go to step 6.
- When supporting only the module version of these drivers, it is possible to run the following to compile patched drivers and to install them into the proper module directory:
% make modules% make modules_installThere is no need to compile the complete kernel. Go to step 8.
Uninstalling the BASP RPM Package
Removing a Physical Interface in Generic Trunking and IEEE 802.3ad Mode
In Generic Trunking and IEEE 802.3ad mode, all of the physical and virtual interfaces belonging to a team have the same MAC address. This MAC address is the same address as that of the first physical interface that is bound to the team. Removing the first physical interface from the team using baspcfg and binding it directly to the protocol could lead to having a duplicate MAC address problem on the network. If the removed physical interface does not participate in any traffic, no problem occurs.
To properly remove a physical interface
- Backup the original team configuration script:
% cp /etc/basp/team-gec /etc/basp/backup-gec
NOTE:
- team-gec is the name of the configuration script.
- backup-gec is the name of the backup script. The name of the backup script must not be prefixed with team-.
- Modify the team configuration script to remove the physical interface.
- Stop the team that is running:
% /etc/basp/baspif /etc/basp/backup-gec stop
% /etc/basp/baspteam /etc/basp/backup-gec del- Restart the team.
% /etc/basp/baspteam /etc/basp/team-gec add
% /etc/basp/baspif /etc/basp/team-gec startInstalling BASP SNMP Agent
This BASP SNMP agent is designed to support the configuration and statistics information pertaining to the BASP driver. The BASP SNMP agent is available in two packaging formats: TAR archive and RPM. Both packages include the same script and MIB files.
Installing BASP SNMP Agent from the TAR Archive
- Uncompress and expand the tar archive:
% tar xvfz baspsnmp-version.tar- Install the TAR archive.
- Copy the getBaspInfo and genBaspTraps script files to the /usr/bin directory.
- Copy the BASP-Config-MIB.txt, BASP-Statistics-MIB.txt and Brcm-BSAPTrap-MIB.txt files to the /usr/share/snmp/mibs directory.
- Locate the snmpd.conf file (it is normally located at: /etc/snmp or /usr/lib/snmp or $HOME/.snmp) and add the following lines to the snmpd.conf file:
pass .1.3.6.1.4.1.4413.1.2.1 /usr/bin/getBaspInfopass .1.3.6.1.4.1.4413.1.2.2.1 /usr/bin/getBaspInfopass .1.3.6.1.4.1.4413.1.2.2.2 /usr/bin/getBaspInfopass .1.3.6.1.4.1.4413.1.2.2.3 /usr/bin/getBaspInfo- Stop the snmpd daemon and restart it:
% /etc/init.d/snmpd stop
% /etc/init.d/snmpd start- Run the genBaspTraps script to allow monitoring of the BASP trap events:
% genBaspTrapsIf BASP trap event monitoring is no longer needed, terminate this script by pressing CTRL+C.
The snmpget and snmpgetnext commands can be used to receive the BASP SNMP objects such as:
% snmpget localhost public BASP-Config-MIB::btTeamNumber% snmpgetnext localhost public BASP-Config-MIB::btTeamNumberBASP SNMP objects are provided in the following text files:
Installing BASP SNMP Agent from the RPM Package
- Install BASP SNMP Agent from the RPM package:
% rpm -i baspsnmp-version.i386.rpmThis modifies the snmpd.conf configuration file to add support for the BASP SNMP agent.
- Follow steps 46 in Installing BASP SNMP Agent from the TAR Archive.
NOTE: The current RPM installation fails to append the additional directives to the snmpd.conf file that are needed to support BASP objects. To modify the snmpd.conf file, see step 3 in Installing BASP SNMP Agent from the TAR Archive.
BASP SNMP Files
Uninstalling BASP SNMP Agent from the RPM Package
Known Problems
IEEE 802.3ad team-member links disconnect and reconnect continuously when they are connected to the HP2524 switch. This is a third-party issue. It is seen only when configuring an IEEE 802.3ad team with greater than 2 members on the system and connecting an HP2524 switch with LACP enabled as passive or active. The HP switch shows an LACP channel being brought up successfully with only 2 members. All other member's links disconnect and reconnect. This does not occur with a Cisco Catalyst 6500.