If you liked my RAID 1 HOWTO, you will sure like this one too. However, this time I had decent hardware. Goal was to make machine with cca. 800 Gb of disk space which is also stable as in: I don't want one failing disk to bring machine down. This goal proved to be correct because in first two days, one of disks failed. Server survived without any loss of data.
For this project I used:
Procedure involves many steps, and some of them must be performed in this order. However, that is not the only possible order. Helpful hints are marked specially.
We had 8 disks, 120 Gb each. Since you can't boot from RAID 5, we also created some RAID 1 volumes (for root /, /usr, and /var and last two for swap). So, layout is:
disk partition content partition content 1 hda1 root / RAID 1 (mirror) hda2 /raid RAID 5 2 hdc1 hdc2 3 hde1 /usr RAID 1 (mirror) hde2 4 hdg1 hdg2 5 hdi1 /var RAID 1 (mirror) hdi2 6 hdl1 hdl2 7 hdn1 swap (not mirrored) hdn2 8 hdo1 swap (not mirrored) hdo2
Hint: As you can see, not all our disks are masters. That shouldn't matter, but take some extra time to make them masters so that you have neatly organized server.
Now, let's do a walk-through:
Hint: Don't leave your CD ROM on secondary channel. It will slow down disk which is on the same channel!
Hint: /dev/MAKEDEV as delivered with makedev-2.3.1-63 Debian package have a bug when creating nodes above hdl. All lines with base=`index kl $unit` should have alternating drive letters instead of kl. Bug about this has been submitted.
Hint: Don't try to build RAID 5 with different sizes of partitions. md won't object, but you will surely loose all your data.
Hint: reboot if you got errors from fdisk like:
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
keep in mind that you will really have to reboot before making raid array in next step.
raiddev /dev/md1 raid-level 1 nr-raid-disks 2 nr-spare-disks 0 chunk-size 4 persistent-superblock 1 device /dev/hda1 raid-disk 0 device /dev/hdc1 raid-disk 1 raiddev /dev/md2 raid-level 1 nr-raid-disks 2 nr-spare-disks 0 chunk-size 4 persistent-superblock 1 device /dev/hde1 raid-disk 0 device /dev/hdg1 raid-disk 1 raiddev /dev/md3 raid-level 1 nr-raid-disks 2 nr-spare-disks 0 chunk-size 4 persistent-superblock 1 device /dev/hdi1 raid-disk 0 device /dev/hdl1 raid-disk 1 raiddev /dev/md4 raid-level 5 nr-raid-disks 8 nr-spare-disks 0 chunk-size 128 persistent-superblock 1 parity-algorithm left-symmetric device /dev/hda2 raid-disk 0 device /dev/hdc2 raid-disk 1 device /dev/hde2 raid-disk 2 device /dev/hdg2 raid-disk 3 device /dev/hdi2 raid-disk 4 device /dev/hdl2 raid-disk 5 device /dev/hdn2 raid-disk 6 device /dev/hdo2 raid-disk 7
Hint: You can speed up things (and avoid some of problems with Seagate drives) if you do proper setup of your disks with hdparm. Here is our script.
Hint: You don't really have to wait for large RAID 5 to re-sync (it takes about 4 hours). However, your performance will be severely degraded if RAID is re-sync-ing while you work.
Hint: You can always use some other method of installation. I started with old vanilla image of Woody for User Mode Linux that I use. It was not worth it. Stick with regular methods.
boot=/dev/md1 raid-extra-boot=/dev/hda,/dev/hdc root=/dev/md1 install=/boot/boot-menu.b map=/boot/map delay=20 vga=normal default=Linux image=/vmlinuz label=Linux read-only initrd=/initrd.img image=/vmlinuz.old label=LinuxOLD read-only optional initrd=/initrd.img.old
Hint: You didn't forgot to apt-get kernel-image which is correct for your machine (e.g. SMP). This step will also make initrd image, with support for RAID.
This is more-or-less all what you have to do. Your RAID will work from now on, and only important command is raidhotadd with which you can add disk if one of them fail.
Failure recovery procedure is: