Changes between Initial Version and Version 1 of 3wareRaid


Ignore:
Timestamp:
12/14/07 11:19:17 (12 years ago)
Author:
joshuadf
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • 3wareRaid

    v1 v1  
     1= 3ware RAID =
     2
     3A local copy of 3ware code, utilities, and documentation is on `bursa:/ISO`.
     43ware downloads are available at http://www.3ware.com/support/download.asp
     5(the CLI and 3DM2 for 9xxx are compatible with 7xxx/8xxx controllers). 3ware
     6has good linux support and the drivers are integrated in the kernel.org tree.
     7
     8In some cases you may need to use the BIOS or older utilities (7.7.x, or
     9`tw_cli help` says `version 2.00.00.032b`)  for functionality specific to the
     10older controllers.
     11
     12The below examples use the newer utilities.
     13
     14
     15== Adding a volume ==
     16
     17If you are replacing all the old drives with new larger ones, first
     18delete the old volume, then remove and replace the drives with the
     19new larger ones:
     20{{{
     21tw_cli maint deleteunit c1 u0
     22# not necessary, but minimizes error messages:
     23tw_cli maint remove c1 p0
     24tw_cli maint remove c1 p1
     25tw_cli maint remove c1 p2
     26tw_cli maint remove c1 p3
     27}}}
     28When the 3ware card sees the new drives, it will add them as
     29individual one-drive "JBOD" units. Delete the single units and
     30create a RAID:
     31{{{
     32tw_cli maint rescan
     33tw_cli maint deleteunit c1 u0
     34tw_cli maint deleteunit c1 u1
     35tw_cli maint deleteunit c1 u2
     36tw_cli maint deleteunit c1 u3
     37tw_cli maint createunit c1 rraid5 p0 p1 p2
     38tw_cli maint createunit c1 rspare p2
     39}}}
     40
     41You now have a new volume, which the OS should see (check `/var/log/messages`).
     42Assuming it is `/dev/sdb`, create a new disk label:
     43{{{
     44# adding a disk:
     45DISKDEV=/dev/sdb
     46PART=${DISKDEV}1
     47parted $DISKDEV mklabel msdos
     48parted $DISKDEV print
     49# get the end block number from above
     50ENDBLOCK=953875
     51}}}
     52And finally create the partition and filesystem and turn off the automatic fsck:
     53{{{
     54parted $DISKDEV mkpart primary 0 953875
     55mke2fs -O dir_index -j $PART
     56tune2fs -c 0 -i 0 $PART
     57}}}
     58
     59
     60== Replacing a drive ==
     61
     62If a RAID unit is `DEGRADED`, a drive had died. Find out which port with
     63`tw_cli info c0`:
     64{{{
     65Unit  UnitType  Status         %Cmpl  Stripe  Size(GB)  Cache  AVerify  IgnECC
     66------------------------------------------------------------------------------
     67u0    RAID-5    DEGRADED       -      64K     701.284   ON     -        -       
     68
     69Port   Status           Unit   Size        Blocks        Serial
     70---------------------------------------------------------------
     71p0     DEGRADED         u0     233.76 GB   490234752     Y61DS7HE     
     72p1     OK               u0     233.76 GB   490234752     Y63H22SE     
     73p2     OK               u0     233.76 GB   490234752     Y61534QE     
     74p3     OK               u0     233.76 GB   490234752     Y61ECSLE     
     75}}}
     76
     77Remove that port from the set:
     78{{{
     79[root@vagal ]# tw_cli maint remove c0 p0
     80Exporting port /c0/p0 ... Done.
     81}}}
     82Then pull and replace the drive (verify the Serial on the drive sticker), rescan, and rebuild.
     83It probably would be best to do this via the 3ware BIOS (`Alt-3` on boot) since last time Linux recognized the new physical drive as `sda`, making a mess of `fstab`.
     84{{{
     85[root@vagal ]# tw_cli maint createunit c0 rspare p0
     86Creating new unit on controller /c0 ...  Done. The new unit is /c0/u0.
     87
     88[root@vagal ]# tw_cli maint rebuild c0 u1 p0
     89Sending Rebuild-Start request to /c0/u1 on 1 disk(s) [0] ... Done.
     90
     91[root@vagal ~]# tw_cli info c0
     92
     93Unit  UnitType  Status         %Cmpl  Stripe  Size(GB)  Cache  AVerify  IgnECC
     94------------------------------------------------------------------------------
     95u1    RAID-5    REBUILDING     18     64K     701.284   ON     -        -       
     96
     97Port   Status           Unit   Size        Blocks        Serial
     98---------------------------------------------------------------
     99p0     DEGRADED         u1     233.76 GB   490234752     Y66G1R3E     
     100p1     OK               u1     233.76 GB   490234752     Y63H22SE     
     101p2     OK               u1     233.76 GB   490234752     Y61534QE     
     102p3     OK               u1     233.76 GB   490234752     Y61ECSLE     
     103
     104}}}
     105
     106Alternatively, if the `DEGRADED` drive isn't really bad (say, you pulled
     107the wrong drive on a replace), you can just rescan the container and rebuild:
     108{{{
     109[root@vagal ]# tw_cli rescan
     110Rescanning controller /c0 for units and drives ...Done.
     111Found the following unit(s): [none].
     112Found the following drive(s): [none].
     113Rescanning controller /c1 for units and drives ...Done.
     114Found the following unit(s): [none].
     115Found the following drive(s): [/c1/p0].
     116
     117
     118[root@vagal ~]# tw_cli maint rebuild c1 u0 p0
     119Sending Rebuild-Start request to /c1/u0 on 1 disk(s) [0] ... Done.
     120
     121
     122[root@vagal ~]# tw_cli info c1
     123
     124Unit  UnitType  Status         %Cmpl  Stripe  Size(GB)  Cache  AVerify  IgnECC
     125------------------------------------------------------------------------------
     126u0    RAID-5    REBUILDING     2      64K     701.284   ON     -        -       
     127
     128Port   Status           Unit   Size        Blocks        Serial
     129---------------------------------------------------------------
     130p0     DEGRADED         u0     233.76 GB   490234752     Y62K8SDE     
     131p1     OK               u0     233.76 GB   490234752     Y61DS8NE     
     132p2     OK               u0     233.76 GB   490234752     Y61DNJKE     
     133p3     OK               u0     233.76 GB   490234752     Y61DS7DE     
     134
     135
     136}}}
     137
     138== vastus example ==
     139
     140For some functionality the older utilities must be used on `vastus`.
     141
     142{{{
     143tw_cli info c0 u0
     144# The "N/A" means the drive is bad:
     145#       Subunit 0:    CBOD: DEGRADED
     146#       Physical Port: N/A
     147tw_cli maint remove c0 p0
     148# Now note the drive serial number (from "info c0") then power down and replace the drive
     149tw_cli maint rescan c0
     150tw_cli maint createunit c0 rspare p0
     151tw_cli maint rebuild c0 u0 p0
     152
     153}}}
     154
     155There is a pretty good example at
     156http://gentoo-wiki.com/HARDWARE_3ware_Raid_Array_Management
     157
     158
     159== 3dmd ==
     160
     161The configuration file for `3dmd` is `/etc/3dmd.conf`. By default it runs on
     162port 1080 and listens to remote connections but refuses access to settings.
     163It is capable of sending email alerts but I have not tested that feature.
     164
     165== The tw_cli (3ware command-line interface) tool ==
     166
     167The User's Guide for the newer `tw_cli` is available from
     168http://www.3ware.com/support/UserDocs/CLIGuide.pdf
     169
     170{{{
     171[root@vagal root]# tw_cli show
     172
     173Ctl   Model        Ports   Drives   Units   NotOpt   RRate   VRate   BBU
     174------------------------------------------------------------------------
     175c0    8506-4LP     4       4        1       0        3       -       - 
     176c1    8506-4LP     4       4        1       0        3       -       - 
     177
     178[root@vagal root]# tw_cli info c0
     179
     180Unit  UnitType  Status         %Cmpl  Stripe  Size(GB)  Cache  AVerify  IgnECC
     181------------------------------------------------------------------------------
     182u0    RAID-5    OK             -      64K     701.284   ON     -        -       
     183
     184Port   Status           Unit   Size        Blocks        Serial
     185---------------------------------------------------------------
     186p0     OK               u0     233.76 GB   490234752     Y61DS7HE     
     187p1     OK               u0     233.76 GB   490234752     Y63H22SE     
     188p2     OK               u0     233.76 GB   490234752     Y61534QE     
     189p3     OK               u0     233.76 GB   490234752     Y61ECSLE     
     190
     191[root@vagal root]# tw_cli info c1
     192
     193Unit  UnitType  Status         %Cmpl  Stripe  Size(GB)  Cache  AVerify  IgnECC
     194------------------------------------------------------------------------------
     195u0    RAID-5    OK             -      64K     701.284   ON     -        -       
     196
     197Port   Status           Unit   Size        Blocks        Serial
     198---------------------------------------------------------------
     199p0     OK               u0     233.76 GB   490234752     Y62K8SDE     
     200p1     OK               u0     233.76 GB   490234752     Y61DS8NE     
     201p2     OK               u0     233.76 GB   490234752     Y61DNJKE     
     202p3     OK               u0     233.76 GB   490234752     Y61DS7DE     
     203}}}
     204
     205
     206The User's Guide for the older `tw_cli` on vastus is available from
     207http://www.3ware.com/support/UserDocs/CLI_UG.pdf
     208
     209{{{
     210vastus:/# tw_cli info c0
     211Controller: c0
     212-------------
     213Driver:   1.02.00.037
     214Model:    7850
     215FW:       FE7X 1.05.00.065
     216BIOS:     BE7X 1.08.00.048
     217Monitor:  ME7X 1.01.00.038
     218Serial #: F11602A2162007
     219PCB:      Rev3
     220PCHIP:    1.30-33
     221ACHIP:    3.20
     222
     223
     224# of units: 1
     225        Unit 0: RAID 5 1.04 TB ( 2241197056 blocks): OK
     226
     227# of ports: 8
     228        Port 0: Maxtor 4G160J8 G8000000 152.67 GB (320173056 blocks): OK(unit 0)
     229        Port 1: Maxtor 4G160J8 G8000000 152.67 GB (320173056 blocks): OK(unit 0)
     230        Port 2: Maxtor 4G160J8 G8000000 152.67 GB (320173056 blocks): OK(unit 0)
     231        Port 3: Maxtor 4G160J8 G8000000 152.67 GB (320173056 blocks): OK(unit 0)
     232        Port 4: Maxtor 4G160J8 G8000000 152.67 GB (320173056 blocks): OK(unit 0)
     233        Port 5: Maxtor 4G160J8 G8000000 152.67 GB (320173056 blocks): OK(unit 0)
     234        Port 6: Maxtor 4G160J8 G80438HE 152.67 GB (320173056 blocks): OK(unit 0)
     235        Port 7: Maxtor 4G160J8 G80EQFYE 152.67 GB (320173056 blocks): OK(unit 0)
     236}}}