IBM Microcode Update Readme File

Firmware Level	Description
0310204100102	Original Release
0310303970033	Adds network boot support and resets the base firmware level. October, 2011
0400003260003	Fixes EEH problem seen with Linux FCode fix for one port (port 1) not being configured (DEFINED), occasionally, especially during HOT PLUG Adds enforcement for supported cable type (10G Fiber-SR or Copper-Twinax)
0400401800007	Fixed link stability issues
0400401800009	Impact: Usability Serverity: ATT Scheduler implemented to handle ioctl commands as a priority to prevent mailbox timeouts. Advanced Mode regression fix for FC5287/5284

Document Revision History	Description
1.0	Initial release to drop code level 0310204100102
1.1	Adding Linux Update instruction
2.0	Release to drop code level 0310303970033
2.1	Adding firmware level 0400003260003
2.2	Adding firmware level 0400401800007
2.3	Section 5.0 OS Requirements content change
2.4	Adding firmware level 0400401800009
2.5	Added information about cable type enforcement in 0400003260003 and newer microcode levels. Revised instructions that mentioned the need to powercycle on linux after microcode update, changed to reboot of lpar needed on linux after microcode update. Adding Impact/Severity to microcode revision history

2.0 General information

This Readme file is intended to give directions on how to update the microcode found on the PCIe2 2-port 10GbE SR Adapter

Device ID for FC 5287, 5284: a21910071410d003.

Device ID for FC 5288, 5286: a21910071410d103.

Key Note #1: This adapter requires an adapter reset after applying new microcode.

If running network traffic, this means executing an LPAR reboot immediately after updating the desired cards, as the adapter will not function properly until reset. Updating the adapter in AIX does not require a reboot if you follow the instructions to temporarily unconfigure and bring the network down on the adapter ports.

Key Note #2: Any microcode level beyond 0310303970033 will enforce the adapter feature code cable type. Please be sure to use the supported 10Gb fiber-optics + transceiver module for FC 5287, 5284, or 10Gb Twinaxial (Copper) cabling for FC 5288, 5286.

3.0 Installation Time

Approximately 20 minutes per adapter.

4.0 Machine's Affected

This adapter is supported in Power Systems that support PCIe Gen2 slots.

5.0 OS Requirements:

5.1 AIX

AIX 61L technology level 6100-06 SP3 and later

AIX 710 technology level 7100-00 SP2 and later

5.2 LINUX

RHEL 6.0 and later

6.0 Determine the Current Microcode Level

6.1 AIX

Before you install the microcode, it is important to determine the microcode level of the Adapter installed in the target system. Use the following instructions to read the ROM level stored in the Adapter's VPD.

1) List the Ethernet adapters installed in the system by typing:

lsdev -C | grep ent

Note: All of the Ethernet adapters that are installed will be displayed. They will be listed as entX, where X is 0, 1, etc.

2) To check the current microcode level for the adapter or controller enter the following command

lscfg -vl entX

Where “x” is the instance of the adapter. The command will produce output similar to:

# lscfg -vl ent4

ent4 U78AA.001.WIH003H-P1-C1-C2-T1 PCIe2 2-port 10GbE SR Adapter(a21910071410d003)

PCIe2 2-port 10GbE SR Adapter, NIC PF:

Part Number.................74Y2094

Serial Number...............YL1025211026

Feature Code/Marketing ID...5287

FRU Number..................74Y2094

EC Level....................D76887

Manufacture ID..............1037

Network Address.............0000C9F235E6

ROM Level.(alterable)....... 0400401800009

Hardware Location Code......U78AA.001.WIH003H-P1-C1-C2-T1

If the ROM Level is less than 04 004 0180 0009 you should update the microcode.

6.2 LINUX

1) Use this series of Linux commands to determine which of any installed Network adapters are Integrated Multifunction Adapters:

Note: It is advisable to cut and paste these command strings into your system console, to avoid typographical errors.
Note: All of the Ethernet adapters that are installed will be displayed. They will be listed as ethX, where X is 0, 1, etc.

for i in `ls /sys/class/net | grep eth` ; do echo -n "${i}: "; ethtool -i ${i} | grep bus-info; done

Output will display the association between ethX devices and PCI bus device numbers, nnnn:nn:nn.n

eth0: bus-info:
eth1: bus-info: 0000:01:00.0
eth2: bus-info: 0000:01:00.1

lspci | grep be3

Output will list PCI bus device numbers for the Integrated Multifunction Adapter functions, nnnn:nn:nn.n

0000:01:00.0 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02)
0000:01:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02)

This example shows the two ports of one adapter. Firmware need only be updated on the first port of each adapter,
eth1 in this case.

If more than 2 ports are shown, it is likely you have more than one adapter, so you would need to also update firmware on first port of each
adapter.

2) Use ethtool –i ethX to find out which Ethernet interface is associated with BE3 adapter. If driver name is “be2net” and the bus-info matches data from the lspci command, that interface is for Integrated Multifunction Card. For example:

#ethtool –i ethX

Where “x” is the instance of the adapter. The command will produce output similar to:

#ethtool –i eth1

driver: be2net
version: 4.1.307r
firmware version: 4.4.180.9
bus-info: 0001:01:00.4

If driver name is “be2net” and the bus-info matches the data from lspci, eth1 is one of Integrated Multifunction Card interfaces. Firmware-version is “4.4.180.9”.

If firmware-version is less than 4.4.180.9 you should update the microcode.

7.0 Downloading the RPM Format File to the Target Server

7.1 AIX

Use this method to download to an AIX system:

NOTE: The instructions that follow are specific AIX commands. AIX commands are CASE (lower and upper) SENSITIVE, and must be entered exactly as shown, including filenames.

1) Transfer the RPM format file to the /tmp directory (using “Save as....”).

2) Install the microcode on your system using the rpm command as follows:

# rpm -ihv --ignoreos /tmp/a21910071410d103.0400401800009.aix.rpm

3) For AIX: The microcode files will be added to /etc/microcode/.

NOTE:
- "/etc/microcode" is a symbolic link to "/usr/lib/microcode".
- If permission does not allow the copy to the above stated directory or file then the user will be prompted for a new location.

7.2 LINUX

Use this method to download the new microcode to a Linux system:

NOTE: The instructions that follow are specific Linux commands. Linux commands are CASE (lower and upper) SENSITIVE, and must be entered exactly as shown, including filenames.

1) Transfer the RPM format file to the /tmp directory (using “Save as....”).

2) Install the microcode on your system using the rpm command as follows:

# rpm -ivh /tmp/a21910071410d103.0400401800009.Linux.rpm

3) The microcode file will be added to /lib/firmware directory. Files name are: a21910071410d103.0400401800009 and a21910071410d003.0400401800009. File contents are identical and any can be used for firmware update on Linux.

8.0 Discovery Tool Microcode CD-ROM creation and download instructions

Microcode Discovery Service is used to determine if microcode installed on your IBM System P systems is at the latest level.

To obtain information how to burn a CD-ROM and run the Discovery Tool for an AIX or Linux System please go to: http://www14.software.ibm.com/webapp/set2/mds/

1) After running the Discovery Tool successfully the "/tmp/microcode/RPM" directory was created and your rpm files are copied from the CD-ROM.

2) Change to that directory, "cd /tmp/microcode/RPM".

3) Unpack the file by executing the instructions below:
Enter the commands:
rpm -ihv --ignoreos a21910071410d103.0400401800009.aix.rpm

4) One microcode file will be copied to "/etc/microcode". The file size and checksum of the microcode image will be verified in Section 9.0.
File Name:
a21910071410d103.0400401800009 and a21910071410d003.0400401800009

9.0 Verifying microcode level before download

9.1 AIX

Please verify the file size and checksum of the raw microcode files matches what is listed below.

For AIX :
"ls -l /etc/microcode/a21910071410d003.0400401800009" to verify file size is 16777392.
"sum /etc/microcode/a21910071410d003.0400401800009" to verify Checksum is 58478.

"ls -l /etc/microcode/a21910071410d103.0400401800009" to verify file size is 16777392.
"sum /etc/microcode/a21910071410d103.0400401800009" to verify Checksum is 58478.

9.2 LINUX

"ls -l /lib/firmware/a21910071410d003.0400401800009" to verify file size is 16777392.
"sum /lib/firmware/a21910071410d003.0400401800009" to verify Checksum is 58478.

"ls -l /lib/firmware/a21910071410d103.0400401800009" to verify file size is 16777392.
"sum /lib/firmware/a21910071410d103.0400401800009" to verify Checksum is 58478.

10.0 Microcode Download Procedure:

Microcode download must be performed separately for each Adapter under Partitions (LPAR's) that owns the adapter.

10.1 AIX

10.1.1 Setting up for Microcode download

1. Stop all applications that use this interface/adapter

2. Remove the interface/IP address from the all ports identified in section 6.0 for the adapters that will be upgraded.

a. Before detaching the interface, record the ip address and any other pertinent information that was configured on the Adapter. This information may be needed if the microcode update overwrites this section on the Adapter.

3. If the interfaces are members of a Shared Ethernet Aadapter (SEA), the SEA devices must be moved to a defined state.

a. “rmdev -l enX” – where “X” is the interface number for the Shared Ethernet Adapter.

b. “rmdev –l entX” – where “X” is the interface number for the Shared Ethernet Adapter.

4. If the interfaces are members of an EtherChannel, the EtherChannel device must be moved to a define state.

a. “rmdev -l enX” – where “X” is the interface number for the EtherChannel adapter.

b. “rmdev –l entX” – where “X” is the interface number for the EtherChannel adapter.

5. For every port associated with the adapter, the enX interfaces must be changed to a defined state.

a. “rmdev -l enX” - where “X” is the interface number for the adapter port

b. This command will be run 2 times for each adapter, once for each port on the adapter.

10.2 Downloading Microcode to the Adapter

1) At the command line type “diag”
2) Select the "Task Selection" from diagnostics menu.
3) Select "Microcode Tasks” then select “Download Microcode" from the menu.
4) Select one entX device for each of the 2-port 10GbE SR Adapter(a21910071410d003) that need to be updated from the list of devices by using the arrow keys to highlight the entry and pressing "Enter" to mark it. The devices will be displayed as ent#, where # is the number of the device. Press "F7" when you are done mark all the adapters you want to flash.
5) Select “/etc/microcode” or "/usr/lib/microcode".
6) A dialogue box may be displayed on screen. It will state that the current microcode level on the adapter is not in the /etc/microcode directory. This is acceptable because the adapter will reject any incorrect code. Press "Enter" to continue.
7) Select 0400401800009 level and press "Enter" to flash the adapter.
8) The following message will appear on the screen when download is completed: "Microcode download complete successfully. The current microcode level for the device ent# is ...
9) If you selected more than one adapter to update, then steps 6-9 will repeat until all adapters are updated.
10) Exit diagnostics.

10.3 Verify and Re-configure adapters

1) Verify the code level is 0400401800009 by typing "lscfg -vl ent#" for each Ethernet adapter updated (where # is the instance of the Ethernet adapter).

2) Re-configure/Verify the adapter’s interface ip information through the “smit” menus.

10.2 LINUX

10.2.1 Setting up for Microcode download

1) Use this series of Linux commands to determine which of any installed Network adapters are Integrated Multifunction Adapters:

Note: It is advisable to cut and paste these command strings into your system console, to avoid typographical errors.

for i in `ls /sys/class/net | grep eth` ; do echo -n "${i}: "; ethtool -i ${i} | grep bus-info; done

Output will display the association between ethX devices and PCI bus device numbers, nnnn:nn:nn.n

eth0: bus-info:
eth1: bus-info: 0000:01:00.0
eth2: bus-info: 0000:01:00.1

lspci | grep be3

Output will list PCI bus device numbers for the Integrated Multifunction Adapter functions, nnnn:nn:nn.n

0000:01:00.0 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02)
0000:01:00.1 Ethernet controller: Emulex Corporation OneConnect 10Gb NIC (be3) (rev 02)

This example shows the two ports of one adapter. Firmware need only be updated on the first port of each adapter, eth1 in this case.

If more than 2 ports are shown, it is likely you have more than one adapter, so you would need to also update firmware on first port of each adapter.

2) Stop all applications that use the interface/adapter to be updated

3) Configure the adapter’s interface, eth1 in this example.

ifconfig eth1 192.168.1.1/24

4) Check that no errors were reported. If errors were reported verify proper interface was configured and command was entered correctly.

10.2.2 Downloading Microcode to the Adapter

Please be sure to reset the adapter after updating microcode. See General Information Key Note #1 in Section 2.0.

1) Using the ethX configured in previous step, use ethtool –flash to update the firmware.
Note: RPM installed three images; the images are identical in content and only different by name.

ethtool -f ethX a21910071410d003.0400401800009

2) If flashing is completed successfully, no messages will appear on the screen. You can check messages in system log.

dmesg | grep be2net

For example, dmesg command will show the following messages:

be2net 0000:01:00.0: firmware: requesting a21910071410d003.0400401800009
be2net 0000:01:00.0: Flashing firmware file a21910071410d003.0400401800009
be2net 0000:01:00.0: Firmware flashed successfully

These messages show flashing is completed successfully.

3) Verify firmware has been activated

Using same ethX interface used for ethtool –flash command, type:

ethtool -i ethX

driver: be2net
version: 4.1.307r
firmware-version: 4.4.180.7 [4.4.180.9]
bus-info: 0000:01:00.0

4) Firmware-version must show 4.4.180.9. If firmware-version is not as expected, carefully retrace steps to confirm no steps were omitted or performed incorrectly. If no errors or omissions were found, contact IBM customer service for assistance.

5) On a non hot-pluggable system/slot where you install adapter, you have to reboot the entire system (not just LPAR reboot or shutdown) for the update to take effect. On hot-pluggable system/slot, LPAR reboot is sufficient for the new FW to load.

10.2.3 Verify and Re-configure adapters

1) Verify the code level is 4.004.180.009 by typing "ethtool –i ethX" for each Ethernet adapter updated. <Where X is the instance of the Ethernet adapter>

2) Re-configure/Verify the adapter’s interface ip by “ifconfig ethX” <where X is the instance of the Ethernet adapter>. Be aware that this adapter requires a slot reset (See Key Note #1 in Section 2.0). It will not function properly until this is performed.