Change your system from standard partitions into a raid array

Introduction

This document shows how to convert a running system with standard partition types, into a system running a Raid 1 array.  The principals in this document can be adapted to those of Raid 5 or other types of raid array.

Please note, this document is based on using the raid functionality within Linux, and not hardware raid.  Of course, hardware raid is much preferred, since it's faster.  However, software raid within Linux is pretty good, and an excellent alternative if you don't want to purchase a hardware raid controller.

This example is using a virtual machine that I created in VMware.  The steps are the same, but I used 2 x 8GB disks for testing and writing the procedure.  My live system is running 2 x 160GB disks and was created using these procedures.

Please note, I won't be held responsible for data loss.  This is your responsibility, not mine, so make sure you have backed up all your important data.

Also, you need to make sure that you have the mdadm package installed, otherwise none of this will work!

Pre-Conversion

This is how my partitions looked like before the conversion:

Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks     Id  System
/dev/sda1               1         123      987966   82  Linux swap / Solaris
/dev/sda2   *         124         136      104422+  83  Linux
/dev/sda3             137         259      987997+  83  Linux
/dev/sda4             260        1044     6305512+   5  Extended
/dev/sda5             260         503     1959898+  83  Linux
/dev/sda6             504        1044     4345551   83  Linux

So that you understand my partition layout, this is what each partition is:

/dev/sda1 = swap
/dev/sda2 = /boot
/dev/sda3 = /
/dev/sda5 = /usr
/dev/sda6 = /var

I've gone for a more complicated setup, because I want to show more flexibility.  The test machine, was effectively a server, which is why /home doesn't exist as being separate, as I wasn't intending to use it as a desktop system.  Normally, a default Mandriva system, would be something like this:

/dev/sda1 = swap
/dev/sda5 = /
/dev/sda6 = /home

of course, this can vary depending on how you partitioned it when you set the machine up.  You need to know this information before you can continue, otherwise you'll never be able to boot your system correctly.  Look at your /etc/fstab file for information on your partitions.

Now, install your second hard disk if you haven't done already.  This should become /dev/sdb if you're using SATA/SCSI and /dev/hdb if it's IDE (depending on which IDE controller you attach it to).  Easiest way to find out is using this command after booting the system normally:

fdisk -l

and it will show all disks attached to the system.  I would recommend disconnecting all USB and externally attached disks while you do this, so that you don't get confused on which hard disk is what.

Once your system is booted normally (don't boot from Live CD, boot system normally), I would suggest one of two things.  Either initiate the runlevel 3 sequence, or stop X.  These can be done as follows:

init 3

or stopping X:

CTRL-ALT-F1
Login to console screen that appears as root
service dm stop
service xfs stop

Installing mdadm

If you don't have this already, make sure you install it.  Check if it's installed:

rpm -qa | grep mdadm

if no results:

urpmi mdadm

and then continue with the following section.

Creating the partitions

First, we need to ensure the disk was detected.  If you do this before, then you can skip this, otherwise, verify using:

fdisk -l

and you should have /dev/sda and /dev/sdb (depending on your hard disks installed).  Now, we have to change the partition types on the first disk, so:

fdisk /dev/sda

and toggle each of the partitions.  As you know, in my example, I have six partitions, one of which is an extended partition (/dev/sda4) of which you do not touch.  This is important.  You only do the partitions I marked out above, so:

/dev/sda1 = 1
/dev/sda2 = 2
/dev/sda3 = 3
/dev/sda5 = 5
/dev/sda6 = 6

and when in fdisk, to toggle the partitions, you do:

t
1
fd

to change to Linux raid auto-detect.  Do this for each of the other partitions, except the extended partition.  To save afterwards and exit, press:

w

and you'll be back at the prompt.  You may find an error message about the partition table being updated next time you reboot because it's currently in use.  Don't worry about this.  The changes we're making is to ensure that the sizes of partitions on /dev/sda is the same as on /dev/sdb.  And don't reboot yet, else we'll have big problems!  This is what my partition table looks like now I made those amendments:

Disk /dev/sda: 8589 MB, 8589934592 bytes
255 heads, 63 sectors/track, 1044 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1         123      987966   fd  Linux raid autodetect
/dev/sda2   *         124         136      104422+  fd  Linux raid autodetect
/dev/sda3             137         259      987997+  fd  Linux raid autodetect
/dev/sda4             260        1044     6305512+   5  Extended
/dev/sda5             260         503     1959898+  fd  Linux raid autodetect
/dev/sda6             504        1044     4345551   fd  Linux raid autodetect

as you can see, I left /dev/sda4 as it was.  You have to, it's normal.  The next step is to make /dev/sdb the same.  You could just use fdisk and do this manually, but it would take a while.  Here is a command to make it almost instantaneous:

sfdisk -d /dev/sda | sfdisk /dev/sdb

neat eh!  Now, ensure it's absolutely identical:

fdisk -l /dev/sda
fdisk -l /dev/sdb

and compare all the numbers.

Creating Arrays

Since /dev/sda is in use, we cannot add this to the array yet, so we will build the array with /dev/sda offline, and only /dev/sdb active.  This is how:

mdadm --create /dev/md0 --level=1 --raid-devices=2 missing /dev/sdb1
mdadm --create /dev/md1 --level=1 --raid-devices=2 missing /dev/sdb2
mdadm --create /dev/md2 --level=1 --raid-devices=2 missing /dev/sdb3
mdadm --create /dev/md3 --level=1 --raid-devices=2 missing /dev/sdb5
mdadm --create /dev/md4 --level=1 --raid-devices=2 missing /dev/sdb6

Now, the filesystems need to be created on /dev/sdb.  In my example here, all mine were ext3, but change this for the appropriate command if you use reiserfs, etc:

mkswap /dev/md0
mke2fs -j /dev/md1
mke2fs -j /dev/md2
mke2fs -j /dev/md3
mke2fs -j /dev/md4

Copying Data

Now, we have to activate those partitions so that we can use them, so:

swapon /dev/md0
mkdir /mnt/array
mount /dev/md2 /mnt/array

It's important to ensure that / is mounted to /mnt/array.  As you know from my example, /dev/sda3 was /, but on /dev/sdb this would be /dev/sdb3, and since it's now in a raid array, we don't access the partition directly as /dev/sdb3, but as /dev/md2.

Now, copy the data across:

cp -dpRx / /mnt/array

and mount the other partitions and copy the data:

mount /dev/md1 /mnt/array/boot
mount /dev/md3 /mnt/array/usr
mount /dev/md4 /mnt/array/var
cp -dpRx /boot/* /mnt/array/boot/
cp -dpRx /usr/* /mnt/array/usr/
cp -dpRx /var/* /mnt/array/var/

Boot Configuration

We need to ensure that we set the system to reboot from the array, otherwise we'll have problems getting our system up and running.  The mount points in /etc/fstab, should be something like this:

/dev/md2                /                       ext3    defaults        1 1
/dev/md1                /boot                   ext3    defaults        1 2
/dev/md3                /usr                    ext3    defaults        1 2
/dev/md4                /var                    ext3    defaults        1 2
/dev/md0                swap                    swap    defaults        0 0

of course, my /etc/fstab has other info too, but I've just put what's relevant for the raid setup.

And copy this file to /mnt/array/etc:

cp -dp /etc/fstab /mnt/array/etc/fstab

Now, edit /boot/grub/grub.conf (sometimes menu.lst) so that it boots from the array as well, here is my entry:

timeout 10
color 2
splashimage (hd0,4)/grub/mdv-grub_splash.xpm.gz
default 0
viewport 3 2 77 22
shade 1

itle Mandriva 2007
kernel (hd0,1)/boot/vmlinuz root=/dev/md2 resume=/dev/md0
initrd (hd0,1)/boot/initrd.img

now, copy this file into /mnt/array:

cp -dp /boot/grub/grub.conf /mnt/array/boot/grub/grub.conf

substitute grub.conf with menu.lst if your system uses this.  We now need a configuration file for the raid array, so create /etc/mdadm.conf:

DEVICE        /dev/sda*
DEVICE        /dev/sdb*
ARRAY        /dev/md0 devices=/dev/sda1,/dev/sdb1
ARRAY        /dev/md1 devices=/dev/sda2,/dev/sdb2
ARRAY        /dev/md2 devices=/dev/sda3,/dev/sdb3
ARRAY        /dev/md3 devices=/dev/sda5,/dev/sdb5
ARRAY        /dev/md4 devices=/dev/sda6,/dev/sdb6
MAILADDR    [email protected]

this is enough to ensure the system will boot successfully once we've successfully completed adding /dev/sda to the array.  Copy this file to /mnt/array/etc as well:

cp -dp /etc/mdadm.conf /mnt/array/etc/mdadm.conf

and the final step is to reinstall the boot loader so that it's written to both the MBR's on each disk:

grub --no-floppy
root (hd0,1)
setup (hd0)
device (hd0) /dev/sdb
root (hd0,1)
setup (hd0)
quit

please note (hd0,1) is pointing to /dev/sda2 as I had /boot as a separate mount point.  Otherwise, you specify your / partition instead.

Finishing Touches

Insert your Mandriva CD1 or DVD and reboot using rescue mode:

linux rescue

exit to the prompt when you get a menu screen of options, we'll do the rest of the steps manually.  First we have to activate the array we created, which means we need a /etc/mdadm.conf (you did this already, but we're booted from CD now!):

DEVICE        /dev/sda*
DEVICE        /dev/sdb*
ARRAY        /dev/md0 devices=missing,/dev/sdb1
ARRAY        /dev/md1 devices=missing,/dev/sdb2
ARRAY        /dev/md2 devices=missing,/dev/sdb3
ARRAY        /dev/md3 devices=missing,/dev/sdb5
ARRAY        /dev/md4 devices=missing,/dev/sdb6
MAILADDR    [email protected]

note again, I didn't put the /dev/sdax partitions, because the disk isn't prepared yet.  Activate the array using:

mdadm --assemble --scan

this will scan the config file you just created and activate the array for the partitions on /dev/sdb.  In case this didn't work, you might need to do this step first:

mknod /dev/md0 b 9 0
mknod /dev/md1 b 9 1
mknod /dev/md2 b 9 2
mknod /dev/md3 b 9 3
mknod /dev/md4 b 9 4

if you need more than this, just add one, ensuring that the last number increases just like the mdx number increases.

Now, mount the partitions:

mkdir /mnt/array
mount /dev/md2 /mnt/array
mount /dev/md1 /mnt/array/boot
mount /dev/md3 /mnt/array/usr
mount /dev/md4 /mnt/array/var

check and verify that all the filesystem looks good, and that we have everything.  Now, let's delete the partitions on /dev/sda, as we need to create these from scratch:

fdisk /dev/sda
d
1
d
2
d
3
d
4
w

you'll see I deleted the extended partition.  It also deleted the 5 and 6 partition too at the same time.  Verify the disk is empty:

fdisk -l /dev/sda

and we'll use the sfdisk command like before to ensure the partition table on /dev/sdb is the same on /dev/sda:

sfdisk -d /dev/sdb | sfdisk /dev/sda

and then verify again with:

fdisk -l /dev/sda
fdisk -l /dev/sdb

and make sure all the numbers match up.  Now, let's add /dev/sda to the array:

mdadm --add /dev/md0 /dev/sda1
mdadm --add /dev/md1 /dev/sda2
mdadm --add /dev/md2 /dev/sda3
mdadm --add /dev/md3 /dev/sda5
mdadm --add /dev/md4 /dev/sda6

they will add, now check the status of the array building process using:

cat /proc/mdstat

and wait until this has finished before continuing.  It can take a while if your disks are large, be patient :-).  Now, we need to chroot the environment to ensure the kernel can boot our system:

mount -t proc proc /mnt/array/proc
chroot /mnt/array /bin/bash
source /etc/profile

Mandriva uses initrd, which is why we need to do these next steps.  Otherwise, you won't be able to boot your system.  This is simple enough.  First, check your /boot directory for the kernel and initrd.  We have to rename the initrd so that we can create a new one.  I'm working with the Mandriva 2007 default kernel of kernel-2.6.17-5mdv and the initrd-2.6.17-5mdv.img files, so:

mv /boot/initrd-2.6.17-5mdv.img /boot/initrd-2.6.17-5mdv.img.old

check your /boot/grub/grub.conf (or menu.lst) file to ensure that you are working with the right kernel - in case you have multiple kernels installed.  Or, you'll have to repeat this process for all of them.  Now, create the initrd:

mkinitrd initrd-2.6.17-5mdv.img 2.6.17-5mdv

the first part is the filename of the initrd that we renamed before, and the second is the kernel version.

After all this has been done, you can safely reboot the system:

exit

umount /mnt/array/proc /mnt/array/boot /mnt/array/usr /mnt/array/var /mnt/array

reboot

or alternatively, replace reboot with:

halt

if you want to shut down the system.  When you boot the system, you'll find you are now booted with the arrays, and all should be working perfectly fine.