Thursday, October 15, 2015

Upgrading Intel SSD firmware on Dell servers

I have retail Intel DC S3710 SSDs installed on Dell servers and and I wanted to do firmware upgrades on them. I went to the product page and found there is an Intel SSD Data Center Tool.

Links as of 10/15/2015 are

download: https://downloadcenter.intel.com/download/23931/Intel-Solid-State-Drive-Data-Center-Tool
manual: https://downloadmirror.intel.com/23931/eng/Intel_SSD_Data_Center_Tool_2_3_x_User_Guide_331961-005.pdf

This is a command line tool. Install the x64 or x32 version. Then run a command prompt as administrator and navigate to c:\isdct

Normally you would run this and expect to get information

C:\isdct>isdct.exe show -intelssd
No results

You will not get information when running it from a Dell server. Dell servers normally have a PERC RAID HBA card between the OS and the drives; PERC is based on the LSI MegaRAID cards.

In the manual, there is this disclaimer:

The Intel SSD DCT does not support SSD Data Center SATA drives behind HBAs (exception: LSI Mega RAID adapters).

So i'm thinking this might work.

The command to enable the LSI adapter is

c:\isdct>isdct.exe set -system EnableLSIAdapter = true
Set EnableLSIAdapter successful.

Results:
1. Dell R610 with a 6/i card

I ran this on a Dell R610 with a 6/i card and then ran the isdct show -intelssd command. After 2 or so minutes, it crashed the server. This can be found in the event logs after rebooting:

Controller event log: Fatal firmware error: Driver detected possible FW hang, halting FW. 

:  Controller 0 (PERC 6/i Integrated)


I couldn't get any further, and had to upgrade the drives on another server and bring them back to this.


2. R710 with a H700 card

I ran it on a R710 with a H700 card and it did complete (although I did get errors in the Wndows System log with any operation)

C:\isdct>isdct show -intelssd
- IntelSSD BTHV503605A9400NGN -
DeviceStatus: Healthy
Firmware: G2010110
FirmwareUpdateAvailable: G2010140
ModelNumber: INTEL SSDSC2BA400G4
ProductFamily: Intel SSD DC S3710 Series
SerialNumber: BTHV503605A9400NGN
Index: 0
DevicePath: LSI4
Bootloader: Property does not exist.

- IntelSSD BTHV50360580400NGN -
DeviceStatus: Healthy
Firmware: G2010110
FirmwareUpdateAvailable: G2010140
ModelNumber: INTEL SSDSC2BA400G4
ProductFamily: Intel SSD DC S3710 Series
SerialNumber: BTHV50360580400NGN
Index: 1
DevicePath: LSI5
Bootloader: Property does not exist.


To run the firmware upgrade you use load and use the index number that it had reported.

C:\isdct>isdct load -intelssd 0
WARNING! You have selected to update the drives firmware!
Proceed with the update? (Y|N): Y
Updating firmware...
Firmware update successful. Please reboot the system.


C:\isdct>

Please note that I lost access to the server after the firmware had completed!

I power cycled the server (server off, wait 5 seconds, turn on) and did the next SSD (after which the server again failed and had to be power cycled). After the reboot I got a good status:

C:\isdct>isdct.exe show -intelssd
- IntelSSD BTHV503605A9400NGN -
DeviceStatus: Healthy
Firmware: G2010140
FirmwareUpdateAvailable: The selected Intel SSD contains current firmware as of
this tool release.
ModelNumber: INTEL SSDSC2BA400G4
ProductFamily: Intel SSD DC S3710 Series
SerialNumber: BTHV503605A9400NGN
Index: 0
DevicePath: LSI4
Bootloader: Property does not exist.

- IntelSSD BTHV50360580400NGN -
DeviceStatus: Healthy
Firmware: G2010140
FirmwareUpdateAvailable: The selected Intel SSD contains current firmware as of
this tool release.
ModelNumber: INTEL SSDSC2BA400G4
ProductFamily: Intel SSD DC S3710 Series
SerialNumber: BTHV50360580400NGN
Index: 1
DevicePath: LSI5

Bootloader: Property does not exist.

Running any isdct operation still showed several Sense and drive not certified errors on the Windows System log though, but they stopped after that.


3. R620 with a PERC H710P Mini

I ran it o a R620 with a PERC H710P Mini

C:\isdct>isdct.exe show -intelssd
- IntelSSD BTHV503605AM400NGN -
DeviceStatus: Healthy
Firmware: G2010110
FirmwareUpdateAvailable: G2010140
ModelNumber: INTEL SSDSC2BA400G4
ProductFamily: Intel SSD DC S3710 Series
SerialNumber: BTHV503605AM400NGN
Index: 0
DevicePath: LSI6
Bootloader: Property does not exist.

- IntelSSD BTHV503602BZ400NGN -
DeviceStatus: Healthy
Firmware: G2010110
FirmwareUpdateAvailable: G2010140
ModelNumber: INTEL SSDSC2BA400G4
ProductFamily: Intel SSD DC S3710 Series
SerialNumber: BTHV503602BZ400NGN
Index: 1
DevicePath: LSI7
Bootloader: Property does not exist.

C:\isdct>isdct load -intelssd 0
WARNING! You have selected to update the drives firmware!
Proceed with the update? (Y|N): y
Updating firmware...
Firmware update successful. Please reboot the system.

C:\isdct>isdct load -intelssd 1
WARNING! You have selected to update the drives firmware!
Proceed with the update? (Y|N): y
Updating firmware...
Firmware update successful. Please reboot the system.

C:\isdct>

This one did not fail on me after the first disk - I was able to run the two firmware upgrades and shut down the server cleanly. I did do power cycle (server off, wait 5 seconds, then turn on) after the firmware upgrades.

4. R610 with a H700 card

I also did this on an R610 with a H700 and the OS failed as well once a SSD was flashed, so it seems to be a problem with the H700 card specifically.