Issue or symptom

In certain cases, Promise VTrak RAID drives that belong to an array may go offline.A sudden power failure that causes a restart of JBODs in “out of sequence order”, may result in drives being marked as 'DEAD'

Also, incorrect shut down or start up order of the VTrak Eclass and Jclass expansion chassis may mark a drive as 'DEAD' Shutting down the VTrak JClass units before the Eclass, or starting the EClass before starting the Jclass will often mark drives as 'DEAD'

In such situations, the array may be marked 'Offline', which results in the the Logical Drive status reported as 'Offline' in the web administration interface. The arrays drives light may be red.

Make sure that the sequence of events is correct, and it is evident in the vtrak event logging. Determine that the drive was some how reported as "Dead". Make sure that we do not have a stale status on the corresponding dead pd's, prior to bringing them back online. Forcing them may result in compromised data. Please contact support for assistance.

Products affected

  • Promise VTrak E-Class RAID subsystem
  • Promise VTrak J-Class expansion chassis

Solution

To resolve this issue, an administrator should manually bring the drives online. To do so, follow these steps:

1. Shutdown and Boot the RAID Head and expansion chassis in the correct order.

 

2. Establish a Telnet connection to the RAID head over port 2300. To do so, open Terminal (in /Applications/Utilities) and issue the following command:

telnet IPAddress 2300

Replace IPAddress with the IP Address assigned to the RAID heads Ethernet port. If you are not certain what IP Address is assigned to the Promise RAID, refer to the Product Manual.

 

3. Authenticate when prompted. If you are not certain what administrator name and password is assigned to the Promise RAID, refer to the Product Manual.

 

4. At the CLI prompt, issue the following command:

spare

This command will display all the spare drives configured in your system.

In this example, physical drives (PdId) 3,4,5 and 6 are configured as spare drives. 

======================================================================
Id  OpStatus  PdId CfgCapacity Revertible Type      DedicatedToArray
======================================================================
0   OK        3    698.49GB    Yes        Dedicated 0
1   OK        4    698.49GB    Yes        Global
2   OK        5    931.32GB    No         Global
3   OK        6    931.32GB    No         Dedicated 0

Important You will need to delete each spare before attempting to bring dead drives online. This step is to prevent any array from rebuilding to a spare drive instead of bringing the dead drive back online.

Please note, that all configured spare PdId, revertible state, type and if dedicated to a specific array. You will need to recreate these spares later.

 

5. For each spare to remove, issue the following command:

spare -a del -i SpareId

Replace SpareId with the ID number of the spare to delete. Note that you need to use the spare number, not the physical drive number. For example, to delete Spare Drive Id1 type the following command: spare -a del -i 1

Once all spares are removed, issue the following command:

spare

This will verify all spares are removed. If the list is empty, you have removed all spares. You should see the output below:  

======================================================================
Id  OpStatus  PdId CfgCapacity Revertible Type      DedicatedToArray
======================================================================
No configured spare drive in the subsystem

Now that all spares are gone, you can bring the dead marked drives back online.

 

6. At the CLI promit, issue the following command: 

phydrv

This command will produce a list of drives and their status. Drives with red lights may be reported with status "Dead".

In this example, Drives 17 and 18 are reported as "Dead":

==========================================================================
PdId Model       Type  CfgCapacity Location       OpStatus   ConfigStatus
==========================================================================
16   ST3750640NS SATA  698.49GB    Encl1 Slot16   OK         Array0 SeqNo4
17   ST3750640NS SATA  698.49GB    Encl2 Slot1    Dead       Array0 SeqNo5
18   ST3750640NS SATA  698.49GB    Encl2 Slot2    Dead       Array0 SeqNo6

 

7. For each of the drives with the OpStatus, Dead, issue the following command:

phydrv -a online -p PdId

Replace PdId with the PdId number which corresponds to the number farthest to the left in the previous steps output. For example, to bring Drive 17 online, the following command would be issued: phydrv -a online -p 17

Once this command has been executed for each drive that is reported Dead, issue the following command again to review the status of the drives:

phydrv

Notice that the drives that were reported offline are now reported as Forced Online.
For example:

==========================================================================
PdId Model       Type  CfgCapacity Location       OpStatus   ConfigStatus
==========================================================================
16   ST3750640NS SATA  698.49GB    Encl1 Slot16   OK         Array0 SeqNo4
17   ST3750640NS SATA  698.49GB    Encl2 Slot1    Forced Onl Array0 SeqNo5
18   ST3750640NS SATA  698.49GB    Encl2 Slot2    Forced Onl Array0 SeqNo6

If a drives OpStatus remains Dead, contact Promise Support.

 

8. If the all drives marked as OK and your array is working properly, please recreate your Spare Drives configuration as noted in Step 4. Please see your VTrak EClass manual for configuration of spare drives.