Implementing sofware RAID 6 on CENTOS 7

Posted: June 17, 2015 in Linux

In short,RAID is way of storing the same data in different places  on multiple hard disks

For more info,please visit https://en.wikipedia.org/wiki/RAID.

RAID allows for hardware-failure without losing data, and sometimes performance-increase due to distributed workload.

I chose to write about RAID 6,because it allows 2 hard disk failures.It requires minimum 4 hard disks

In a RAID 6 array with four disks, data blocks will be distributed across the drives, with two disks being used to store each data block, and two being used to store parity blocks.

In case of six disks (2 GB each),8 GB will be available for storage data,2 disks are always dedicated for protection,no matter how much hard disks are included in array

Again,i used KVM virtual machine with 1 system HDD,and additional 4 SATA disks (2 GB each),

which will be used for creating RAID

Untitled

After starting this VM,let’s confirm disks are initialized:

from terminal,type lsblk -d command:

Untitled1

Now,we need to format all hard drives:

gdisk /dev/sda

Assign GPT as the partition table to the disk using the o command,confirm with Y,
now press N to crete partition,because we will use all space available,and default filesystem (ext4) press enter when asked for first and last sector and Hex Code,
press W to write changes,Y to confirm

[root@localhost ja]# gdisk /dev/sda
GPT fdisk (gdisk) version 0.8.6

Partition table scan:
MBR: not present
BSD: not present
APM: not present
GPT: not present

Creating new GPT entries.

Command (? for help): o
This option deletes all partitions and creates a new protective MBR.
Proceed? (Y/N): y

Command (? for help): n
Partition number (1-128, default 1):
First sector (34-4194270, default = 2048) or {+-}size{KMGTP}:
Last sector (2048-4194270, default = 4194270) or {+-}size{KMGTP}:
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300):
Changed type of partition to 'Linux filesystem'

Command (? for help): w

Final checks complete. About to write GPT data. THIS WILL OVERWRITE EXISTING
PARTITIONS!!

Do you want to proceed? (Y/N): y
OK; writing new GUID partition table (GPT) to /dev/sda.
The operation has completed successfully.

Repeat same procedure for remaining disks.

After creating all the partitions check if we did it well:

Untitled4

Creating RAID6 Device

Now,after creating partitions,we can create RAID:

mdadm --create /dev/md6 --level=6 --raid-devices=4 /dev/sd[a-d]1

–create /dev/md6    RAID device namd /dev/md6  will be created

–level=6                  create raid 6

–raid-devices=4       raid consists from 4 disks /dev/sda,/dev/sdb,dev/sdc and /dev/sdd,1 stands for tands for the first partition on each disk

Now,verify raid configuration:

[root@localhost ja]# mdadm -E /dev/sd[a-d]1
/dev/sda1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 3235b46a:18b554e3:2e36e760:c898271a
Name : localhost.localdomain:6  (local to host localhost.localdomain)
Creation Time : Tue Jun 17 11:51:49 2015
Raid Level : raid6
Raid Devices : 4

Avail Dev Size : 4190175 (2046.33 MiB 2145.37 MB)
Array Size : 4189184 (4.00 GiB 4.29 GB)
Used Dev Size : 4189184 (2045.84 MiB 2144.86 MB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=991 sectors
State : clean
Device UUID : 8a1e81bf:38e71ac6:5f9d4011:61dcdcdb

Update Time : Tue Jun 17 11:52:09 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : a28d6360 - correct
Events : 17

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 0
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdb1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 3235b46a:18b554e3:2e36e760:c898271a
Name : localhost.localdomain:6  (local to host localhost.localdomain)
Creation Time : Tue Jun 17 11:51:49 2015
Raid Level : raid6
Raid Devices : 4

Avail Dev Size : 4190175 (2046.33 MiB 2145.37 MB)
Array Size : 4189184 (4.00 GiB 4.29 GB)
Used Dev Size : 4189184 (2045.84 MiB 2144.86 MB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=991 sectors
State : clean
Device UUID : 5a9358c0:5f643012:38b3e264:63c7bc41

Update Time : Tue Jun 17 11:52:09 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : a8fc5632 - correct
Events : 17

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 1
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdc1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 3235b46a:18b554e3:2e36e760:c898271a
Name : localhost.localdomain:6  (local to host localhost.localdomain)
Creation Time : Tue Jun 17 11:51:49 2015
Raid Level : raid6
Raid Devices : 4

Avail Dev Size : 4190175 (2046.33 MiB 2145.37 MB)
Array Size : 4189184 (4.00 GiB 4.29 GB)
Used Dev Size : 4189184 (2045.84 MiB 2144.86 MB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=991 sectors
State : clean
Device UUID : 285e93ad:942a4cac:75b3284c:9469476a

Update Time : Tue Jun 17 11:52:09 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : 402389a5 - correct
Events : 17

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 2
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)
/dev/sdd1:
Magic : a92b4efc
Version : 1.2
Feature Map : 0x0
Array UUID : 3235b46a:18b554e3:2e36e760:c898271a
Name : localhost.localdomain:6  (local to host localhost.localdomain)
Creation Time : Tue Jun 17 11:51:49 2015
Raid Level : raid6
Raid Devices : 4

Avail Dev Size : 4190175 (2046.33 MiB 2145.37 MB)
Array Size : 4189184 (4.00 GiB 4.29 GB)
Used Dev Size : 4189184 (2045.84 MiB 2144.86 MB)
Data Offset : 2048 sectors
Super Offset : 8 sectors
Unused Space : before=1960 sectors, after=991 sectors
State : clean
Device UUID : 36187a4c:57ffdaba:c2ac043c:28ac5f59

Update Time : Tue Jun 17 11:52:09 2015
Bad Block Log : 512 entries available at offset 72 sectors
Checksum : cc8d5457 - correct
Events : 17

Layout : left-symmetric
Chunk Size : 512K

Device Role : Active device 3
Array State : AAAA ('A' == active, '.' == missing, 'R' == replacing)

[root@localhost ja]# mdadm --detail /dev/md6
/dev/md6:
Version : 1.2
Creation Time : Tue Jun 17 11:51:49 2015
Raid Level : raid6
Array Size : 4189184 (4.00 GiB 4.29 GB)
Used Dev Size : 2094592 (2045.84 MiB 2144.86 MB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent

Update Time : Tue Jun 17 11:52:09 2015
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 512K

Name : localhost.localdomain:6  (local to host localhost.localdomain)
UUID : 3235b46a:18b554e3:2e36e760:c898271a
Events : 17

Number   Major   Minor   RaidDevice State
0       8        1        0      active sync   /dev/sda1
1       8       17        1      active sync   /dev/sdb1
2       8       33        2      active sync   /dev/sdc1
3       8       49        3      active sync   /dev/sdd1

Format newly created RAID device (/dev/md6):

[root@localhost ja]# mkfs.ext4 /dev/md6
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=256 blocks
262144 inodes, 1047296 blocks
52364 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1073741824
32 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

Rechek configuration:

cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md6 : active raid6 sdd1[3] sdc1[2] sdb1[1] sda1[0]
4189184 blocks super 1.2 level 6, 512k chunk, algorithm 2 [4/4]

Now create folder in which RAID device /dev/md6 will be mounted:

mkdir /mnt/myraid6

Edit /etc/fstab file to automount RAID device on startup to /mnt/myraid6 folder,add new line to the end:

/dev/md6                              /mnt/myraid6                  ext4               defaults     0      1

/dev/md6-device to be mounted

/mnt/myraid6-where to mount

ext4  – filesystem

defaults -mount options (use: rw, suid, dev, exec, auto, nouser, and async)

0 – option is a binary value (“0” for false and “1” for true) for “dumping.”  This is a pretty much out-dated method of backup for cases when the system went down.  You should leave this as “0”.

0 – The last option is a numeric value for “passing.”  This tells the system the order in which to fsck to perform a file system check.  If a disk has an option of “0” it will be skipped.The root file system should always be “1” and other file systems can go afterward.This works best for journaling file systems like ext3/4 and ReiserFS.

Check /etc/fstab for errors:

mount -av

Untitled5

As message stated,i need to label mount folder in order to avoid SELinux denial
touch /mnt/myraid6 /.autorelabel
Now create some data and test RAID resilience:

echo $(date +%d-%b-%H_%M)>/mnt/myraid6/date

Now reboot the server,after restart simulate HDD error:

mdadm -f /dev/md6 /dev/sda1

Untitled9

Check configuration:

[root@localhost myraid6]# mdadm --detail /dev/md6
/dev/md6:
Version : 1.2
Creation Time : Wed Jun 17 11:21:20 2015
Raid Level : raid6
Array Size : 4189184 (4.00 GiB 4.29 GB)
Used Dev Size : 2094592 (2045.84 MiB 2144.86 MB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent

Update Time : Wed Jun 17 11:55:46 2015
State : clean, degraded
Active Devices : 3
Working Devices : 3
Failed Devices : 1
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 512K

Name : localhost.localdomain:6 (local to host localhost.localdomain)
UUID : 00358bb8:7a4f76f2:09163599:9755b768
Events : 19

Number Major Minor RaidDevice State
0 0 0 0 removed
1 8 17 1 active sync /dev/sdb1
2 8 33 2 active sync /dev/sdc1
3 8 49 3 active sync /dev/sdd1

0 8 1 - faulty /dev/sda1

Disable another HDD

mdadm -f /dev/md6 /dev/sdb1

Untitled11

Check again:
mdadm --detail /dev/md6
Untitled12

Now we have 2 faulty hard disks,but we still can access data mounted on myraid6 folder

Untitled10

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s