PCIe3 40GbE RoCE Converged Host Bus Adapter (FC: EC3A,
EC3B)
******* PLEASE READ THIS ENTIRE NOTICE *********
DATE:
February 6, 2017
Table of
Contents
1.0 Microcode and Document Revision History
7.0
Determine the Current Microcode Level for AIX
8.0 Downloading
the RPM Format File to the Target Server for AIX
9.0 Discovery Tool Microcode CD-ROM
creation and download instructions
10.0 Verifying microcode before
download and notes for AIX
11.0 Microcode Download Procedure for AIX
11.1
Setting up for Microcode download
11.2
Downloading Microcode to the Adapter
11.3 Verify and Re-configure
Adapters
1.0 Microcode
and Document Revision History
|
Firmware Level |
Description |
|
000200405030 / 2.40.5030 |
Impact: Usability Severity: ATT - Fixed an issue where outbound completions were
blocked during driver restart |
|
000200365000 / 2.36.5000 |
1. Fixed length calculated of UDP
packets, which resulted incorrect UPD length causing miscalculation of CRC 2. Fixed completion error issue
when ECN was enabled. |
|
000200355100 / 2.35.5100 |
Firmware change log: |
|
000200325100 / 2.32.5100 |
Original Release |
The Firmware Levels Below Are No Longer Supported By IBM
000200325100 / 2.32.5100
000200325100 / 2.32.5100
It is best practices to update to the latest FW level not only for IBM support
of these products, but for optimal performance and to ensure that all of the
required HW/FW fixes are installed. Once new FW has been released to the field,
we will provide a 6 month grace period for customers to update these products
to the currently supported FW level
Please Update To The
Latest Level At Your Earliest Convenience
|
Document Revision History |
Description |
|
V3.0 – 2/6/2017 |
2.40.5030 |
|
V2.0 – 9/19/2016 |
2.36.5000 |
|
V2.0 – 12/16/2015 |
2.35.5100 |
|
V1.0 – 12/15/2014 |
Original Release |
|
|
|
This Readme file is intended to
give directions on how to update the microcode found on the PCIe3 2-Port 40GbE
RoCE adapter (FC: EC3A and EC3B).
Non-Concurrent Download
The microcode installation does
NOT support concurrent download. The entX devices can
be used during and after the download, but update will not go into effect until
either the device is reloaded using rmdev/cfgmgr or a reboot is performed.
NOTE: It is
recommended that the installation be scheduled during a maintenance window or
during non-peak production periods.
Approximately
20 minutes.
Feature Code EC3A:
8247-21L
8247-22L
8284-22A
8335-GCA
8335-GTA
8348-21C
9119-MHE
9119-MME
Feature
Code EC3B:
8247-42L
8286-41A
8286-42A
8335-GCA
8408-E8E
9119-MHE
9119-MME
To burn FW for
Mellanox ConnectX adapters here are the instructions
(for Linux):
1. Find the PCI slot
the ConnectX-3 adapter is plugged into. Issue the following command:
lspci | grep "ConnectX-3"
For example:
# lspci
| grep "ConnectX-3"
0001:01:00.0 Ethernet controller:
Mellanox Technologies MT27500 Family [ConnectX-3]
This tells us that
the adapter is in the PCIe slot 0001:01:00.0 which
will be needed in the following steps.
2. Start MFT tools by
running “mst start”
#mst start
Starting MST (Mellanox Software Tools)
driver set
Loading MST PCI module - Success
Loading MST PCI configuration module -
Success
Create devices
If the system does
not have the mst command
available, please install the Mellanox Firmware Tools (MFT) available here: http://www.mellanox.com/page/management_tools
3. Verify the current
Firmware level by running “flint -d <pci bus ID> -q”. Replace <pci
bus ID> with the ID found in the lscpi command
from step 2.
# flint -d
0001:01:00.0 q
Image type: FS2
FW Version: 2.36.5000
FW Release Date: 3.9.2015
Rom Info: type=FCODE version=0.0.130 devid=4099 proto=ETH
Device ID: 4099
Description: Node Port1 Port2 Sys image
GUIDs: ffffffffffffffff
ffffffffffffffff ffffffffffffffff
ffffffffffffffff
MACs:
f452146cf200 f452146cf201
VSD:
PSID: IBM1560110023
This
command reveals the current version of the FW; which is 2.35.5100 in this
particular case. If the version is less
than 2.40.5030, please update.
Please
note the value for PSID. This will
determine which image is needed in later steps.
4. Download the
firmware RPM file to the destination machine.
Use
this method to download the new microcode to a Linux system:
NOTE:
The instructions that follow are specific Linux commands. Linux commands are
CASE (lower and upper) SENSITIVE, and must be entered exactly as
shown, including filenames.
rpm -ivh /tmp/b31503101410b504 000200405030.Linux.rpm
The
microcode package will install the firmware images in the /lib/firmware rpm –e
directory. If a message is displayed saying the "package <package_name> is already installed", you will need
to uninstall the listed rpm package. On the command line type:
rpm -e <package_name>
where <package_name> is the name of the package that was returned
in the message. Return to Step 2 and attempt to install the file again.
5. Verify the contents of the
image before flashing:
ls -l
/lib/firmware/b31503101410b504.000200405030 to verify file size:
·
b31503101410b504
000200405030 = 892292
sum
/lib/firmware/b31503101410b504.000200405030 to verify Checksum:
·
b31503101410b504.000200405030
= 42767
6. Update the FW with
"flint -d <pci bus id> -i <image filename> burn"
#
flint -d 0001:01:00.0 -i b31503101410b504.000200405030 burn
Current FW version on
flash: 2.36.5000
New FW version: 2.40.5030
Burning FW image without signatures - OK
Restoring signature
- OK
If you receive the following error message, it means the wrong file was
selected for update, and you need to choose the other filename.
7. At this point we have
burned the new FW on to the Mellanox adapter. However, this FW will not take
effect till the system is rebooted.
Reloading the drivers
is not enough; it requires the system to be rebooted.
Basic card function is supported
on:
· AIX Version 6.1 with the 6100-09
Technology Level and service pack 5 and APAR IV68443 or later
· AIX Version 6.1 with the 6100-08
Technology Level and service Pack 7 or later (planned availability 9/30/2015)
· AIX Version 7.1 with the 7100-03
Technology Level and Service Pack 5 and APAR IV68444 or later
·
AIX Version 7.1 with the 7100-02
Technology Level and Service Pack 7 or later (planned availability
9/30/2015)
If you are using another release
of AIX, ensure that the adapter is supported on that release before you install
the adapter. Contact service and support for assistance.
7.0 Determine the Current Microcode
Level for AIX
Before you install the microcode,
it is important to determine the microcode level of the Adapter installed in
the target system. Use the following instructions to read the ROM level stored
in the Adapter's VPD.
A. List the RoCE adapters installed
in the system by typing:
lsdev | grep
RoCE
B. To check the current microcode
level for the adapter or controller enter the following command:
lsmcode -cd entX
Where "X" is the
instance of the adapter. The command will produce output similar to:
# lsmcode -cd ent4
The current microcode level
for ent4 is 000200405030
If the ROM Level is less
than 000200365000 you should update the microcode.
8.0 Downloading the RPM
Format File to the Target Server for AIX
Use this
method to download to an AIX system:
NOTE: The instructions that follow are
specific AIX commands. AIX commands are CASE (lower and upper) SENSITIVE, and must
be entered exactly as shown, including filenames.
A. Make two directories on your AIX
system to receive the RPM format file.
Enter: "mkdir /tmp/microcode"
and then create this directory
Enter: "mkdir /tmp/microcode/RPM"
B. Transfer the RPM format file to
the /tmp/microcode/RPM directory (using "Save as
..."). Change to that directory, "cd /tmp/microcode/RPM".
You'll see the filename for the RPM file.
"rpm -ihv --ignoreos b31503101410b504.000200405030.aix.rpm"
C. For
AIX: The microcode files will be added to /etc/microcode/.
NOTE:
- "/etc/microcode" is a symbolic link
to "/usr/lib/microcode".
- If permission does not allow the copy to the above stated directory or
file then the user will be prompted for a
new location.
9.0 Discovery Tool Microcode CD-ROM
creation and download instructions
To obtain information how to burn
a CD-ROM and run the Discovery Tool for an AIX or Linux System please go to:
http://www-304.ibm.com/webapp/set2/firmware/lgjsn?mode=10&page=cdrom.html
A. After running the Discovery Tool
successfully the "/tmp/microcode/RPM"
directory was created and your rpm files are copied from the CD-ROM.
B. Change to that directory,
"cd /tmp/microcode/RPM".
C. Unpack the file by executing the
instructions below:
Enter the command:
"rpm -ihv --ignoreos b31503101410b504.000200405030.aix.rpm "
D. One microcode file will be copied to "/etc/microcode".
The file size and checksum of the microcode image will be verified in
Section 10.0.
File Names:
b31503101410b504.000200405030
NOTE:
- "/etc/microcode"
is a symbolic link to "/usr/lib/microcode".
- If permission does not allow
the copy to the above stated directory or file then the user will be prompted
for a new location.
- For customers using the AIX
Diagnostics CD, please refer to the IBM System Hardware information Center for
instructions.
10.0 Verifying microcode before
download and notes for AIX
Please verify the file size and
checksum of the raw microcode files matches what is listed below.
ls -l /etc/microcode/b31503101410b504.000200405030
to verify file size:
·
b31503101410b504.000200405030 = 892292
sum /etc/microcode/b31503101410b504.000200405030 to verify Checksum:
·
b31503101410b504.000200405030
= 42767
11.0 Microcode Download Procedure for AIX
11.1 Setting up for Microcode
download
A. Stop all applications that use
this interface/adapter.
B. Remove the interface/IP address
from the all ports identified in section 7.0 for the adapters that will be
upgraded.
a. Before detaching the interface,
record the IP address and any other pertinent information that was configured
on the Adapter. This information may be
needed if the microcode update overwrites this section on the Adapter.
C. If the interfaces are members of
an SEA, the SEA devices must be moved to a defined state.
a. "rmdev
-l enX" - where "X" is the interface
number for the Shared Ethernet Adapter.
b. "rmdev
-l entX" - where "X" is the interface
number for the Shared Ethernet Adapter.
D. If the interfaces are members of an EtherChannel, the EtherChannel
device must be moved to a define state.
a. "rmdev
-l enX" - where "X” is the interface number
for the EtherChannel adapter.
b. "rmdev
-l entX" - where "X" is the interface
number for the EtherChannel adapter.
E. For every port associated with
the adapter, the enX interfaces must be changed to a
defined state.
a. "rmdev
-l enX" - where "X" is the interface
number for the adapter port.
b. This command will be run 2 times,
once for each port on the adapter.
11.2 Downloading Microcode to the
Adapter
A. At the command line type "diag"
B. Select the "Task
Selection" from diagnostics menu.
C. Select "Microcode Tasks”
then select “Download Microcode" from the menu.
D. Select all of the entX RoCE Converged Network Adapters that
need to be updated from the list of devices by using the arrow keys to
highlight the entry and pressing "Enter" to mark it. Press
"F7" or "ESC+7" when you are done marking all the adapters
you want to flash.
E. If a source selection menu is
displayed, Select "/etc/microcode".
F. A dialogue box may be displayed
on screen. It will state that the current microcode level on the adapter
is not in the /etc/microcode directory. This is
acceptable because the adapter will reject any incorrect code. Press
"Enter" to continue.
G. Select 000200355100 level
and press "Enter" to flash the adapter.
H. The following message will appear
on the screen when download is completed: "Microcode download complete
successfully. The current microcode level for the device is ...
Please run diagnostics on the adapter to ensure that it is functioning
properly."
I.
If you
selected more than one adapter to update, then steps 6-9 will repeat until all
adapters are updated.
J.
Exit
diagnostics.
11.3 Re-configure and Verify adapters
A. Run "cfgmgr"
to reconfigure the adapters that were moved to defined before the update.
B. Verify the code level is 000200405030
by typing "lsmcode -cd entX" for each adapter updated, where
"X" is the instance of the RoCE adapter.