CentOS7使用mdadm组软RAID0-荒岛

这个演示环境模拟于哪种真实场景？

某台机器硬盘容量不够了，要加硬盘了，而且要加2块以上，并且对读写性能要求比较高，对数据安全要求不是特别在意，譬如刷PT的机器。

首先安装mdadm：

yum -y install mdadm

查看我们的硬盘状态，可以看到两块船新的vdb/vdc躺在这里（本文会多次穿插使用lsblk命令，目的只为一个：能够让你清楚的知道每一个环节都对硬盘做了啥。）

[root@softraid ~]# lsblk
NAME                     MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0                       11:0    1  918M  0 rom  
vda                      252:0    0   10G  0 disk 
├─vda1                   252:1    0  488M  0 part /boot
└─vda2                   252:2    0  9.5G  0 part 
  ├─centos_softraid-root 253:0    0  8.6G  0 lvm  /
  └─centos_softraid-swap 253:1    0  976M  0 lvm  [SWAP]
vdb                      252:16   0   10G  0 disk 
vdc                      252:32   0   10G  0 disk

给vdb/vdc创建RAID分区：

fdisk /dev/vdb
fdisk /dev/vdc

流程如下：

[root@softraid ~]# fdisk /dev/vdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x12375e20.

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-20971519, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519): 
Using default value 20971519
Partition 1 of type Linux and of size 10 GiB is set

Command (m for help): p

Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x12375e20

   Device Boot      Start         End      Blocks   Id  System
/dev/vdb1            2048    20971519    10484736   83  Linux

Command (m for help): L

 0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris        
 1  FAT12           27  Hidden NTFS Win 82  Linux swap / So c1  DRDOS/sec (FAT-
 2  XENIX root      39  Plan 9          83  Linux           c4  DRDOS/sec (FAT-
 3  XENIX usr       3c  PartitionMagic  84  OS/2 hidden C:  c6  DRDOS/sec (FAT-
 4  FAT16 <32M      40  Venix 80286     85  Linux extended  c7  Syrinx         
 5  Extended        41  PPC PReP Boot   86  NTFS volume set da  Non-FS data    
 6  FAT16           42  SFS             87  NTFS volume set db  CP/M / CTOS / .
 7  HPFS/NTFS/exFAT 4d  QNX4.x          88  Linux plaintext de  Dell Utility   
 8  AIX             4e  QNX4.x 2nd part 8e  Linux LVM       df  BootIt         
 9  AIX bootable    4f  QNX4.x 3rd part 93  Amoeba          e1  DOS access     
 a  OS/2 Boot Manag 50  OnTrack DM      94  Amoeba BBT      e3  DOS R/O        
 b  W95 FAT32       51  OnTrack DM6 Aux 9f  BSD/OS          e4  SpeedStor      
 c  W95 FAT32 (LBA) 52  CP/M            a0  IBM Thinkpad hi eb  BeOS fs        
 e  W95 FAT16 (LBA) 53  OnTrack DM6 Aux a5  FreeBSD         ee  GPT            
 f  W95 Ext'd (LBA) 54  OnTrackDM6      a6  OpenBSD         ef  EFI (FAT-12/16/
10  OPUS            55  EZ-Drive        a7  NeXTSTEP        f0  Linux/PA-RISC b
11  Hidden FAT12    56  Golden Bow      a8  Darwin UFS      f1  SpeedStor      
12  Compaq diagnost 5c  Priam Edisk     a9  NetBSD          f4  SpeedStor      
14  Hidden FAT16 <3 61  SpeedStor       ab  Darwin boot     f2  DOS secondary  
16  Hidden FAT16    63  GNU HURD or Sys af  HFS / HFS+      fb  VMware VMFS    
17  Hidden HPFS/NTF 64  Novell Netware  b7  BSDI fs         fc  VMware VMKCORE 
18  AST SmartSleep  65  Novell Netware  b8  BSDI swap       fd  Linux raid auto
1b  Hidden W95 FAT3 70  DiskSecure Mult bb  Boot Wizard hid fe  LANstep        
1c  Hidden W95 FAT3 75  PC/IX           be  Solaris boot    ff  BBT            
1e  Hidden W95 FAT1 80  Old Minix      

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'

Command (m for help): p

Disk /dev/vdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x12375e20

   Device Boot      Start         End      Blocks   Id  System
/dev/vdb1            2048    20971519    10484736   fd  Linux raid autodetect

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

再次查看一下硬盘的信息，可以看到两块硬盘有了vdb1/vdc1分区：

[root@softraid ~]# lsblk
NAME                     MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0                       11:0    1  918M  0 rom  
vda                      252:0    0   10G  0 disk 
├─vda1                   252:1    0  488M  0 part /boot
└─vda2                   252:2    0  9.5G  0 part 
  ├─centos_softraid-root 253:0    0  8.6G  0 lvm  /
  └─centos_softraid-swap 253:1    0  976M  0 lvm  [SWAP]
vdb                      252:16   0   10G  0 disk 
└─vdb1                   252:17   0   10G  0 part 
vdc                      252:32   0   10G  0 disk 
└─vdc1                   252:33   0   10G  0 part

现在就可以创建RAID0了：

mdadm -C /dev/md0 -l raid0 -n 2 /dev/vd[b-c]1

这条命令等同于上面的命令，执行是一样的效果，可以说上面的命令是简写，而这是详细：

mdadm --create /dev/md0 --level=raid0 --raid-devices=2 /dev/vdb1 /dev/vdc1

创建完成之后，检查RAID状态：

mdadm --detail /dev/md0

主要看看State/ActiveDevices/WorkingDevices/FailedDevices，没问题就行：

/dev/md0:
           Version : 1.2
     Creation Time : Sat Jun 22 22:43:47 2019
        Raid Level : raid0
        Array Size : 20951040 (19.98 GiB 21.45 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Sat Jun 22 22:43:47 2019
             State : clean 
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

        Chunk Size : 512K

Consistency Policy : none

              Name : softraid:0  (local to host softraid)
              UUID : aba3285e:6ecbfafd:df3daac6:712acf24
            Events : 0

    Number   Major   Minor   RaidDevice State
       0     252       17        0      active sync   /dev/vdb1
       1     252       33        1      active sync   /dev/vdc1

此时我们再次查看硬盘信息，可以看到在分区下有了两个RAID设备：

[root@softraid ~]# lsblk
NAME                     MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sr0                       11:0    1  918M  0 rom   
vda                      252:0    0   10G  0 disk  
├─vda1                   252:1    0  488M  0 part  /boot
└─vda2                   252:2    0  9.5G  0 part  
  ├─centos_softraid-root 253:0    0  8.6G  0 lvm   /
  └─centos_softraid-swap 253:1    0  976M  0 lvm   [SWAP]
vdb                      252:16   0   10G  0 disk  
└─vdb1                   252:17   0   10G  0 part  
  └─md0                    9:0    0   20G  0 raid0 
vdc                      252:32   0   10G  0 disk  
└─vdc1                   252:33   0   10G  0 part  
  └─md0                    9:0    0   20G  0 raid0

现在我们就可以使用LVM创建逻辑卷然后做文件系统（如果不想使用LVM直接把RAID设备做成文件系统也是可以的，不过我不推荐）首先要创建PV：

pvcreate /dev/md0

接着创建VG：

vgcreate imlala-vg /dev/md0

最后创建LV：

lvcreate -l 100%FREE -n imlala-lv imlala-vg

查看LV的详细信息：

lvdisplay

回显得知完整的LV卷名：

[root@softraid ~]# lvdisplay
--- Logical volume ---
LV Path                /dev/imlala-vg/imlala-lv
LV Name                imlala-lv
VG Name                imlala-vg
LV UUID                h5iMee-a49a-7len-V5dJ-XDky-qY2K-dqPSfN
LV Write Access        read/write
LV Creation host, time softraid, 2019-06-22 23:17:01 +0800
LV Status              available
# open                 0
LV Size                <19.98 GiB
Current LE             5114
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     4096
Block device           253:2

创建XFS文件系统：

mkfs.xfs /dev/imlala-vg/imlala-lv

挂载文件系统：

mkdir -p /imlala-data
mount /dev/imlala-vg/imlala-lv /imlala-data

确认挂载状态：

[root@softraid ~]# df -hT
Filesystem                        Type      Size  Used Avail Use% Mounted on
/dev/mapper/centos_softraid-root  xfs       8.6G  1.2G  7.5G  14% /
devtmpfs                          devtmpfs  3.9G     0  3.9G   0% /dev
tmpfs                             tmpfs     3.9G     0  3.9G   0% /dev/shm
tmpfs                             tmpfs     3.9G  8.6M  3.9G   1% /run
tmpfs                             tmpfs     3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/vda1                         xfs       485M  181M  304M  38% /boot
tmpfs                             tmpfs     783M     0  783M   0% /run/user/0
/dev/mapper/imlala--vg-imlala--lv xfs        20G   33M   20G   1% /imlala-data

挂载没问题后，加到fstab：

echo "/dev/imlala-vg/imlala-lv /imlala-data xfs defaults 0 0" >> /etc/fstab

如果又加了硬盘，如何给现有阵列扩容？这里按常理来说，RAID0是不支持扩容的，但我们可以先将阵列转换为RAID4，然后再转换回来，变相的给RAID0阵列做个扩容。

首先看看我们当前的硬盘信息，可以看到新加了一块vdd：

[root@softraid ~]# lsblk
NAME                        MAJ:MIN RM  SIZE RO TYPE  MOUNTPOINT
sr0                          11:0    1  918M  0 rom   
vda                         252:0    0   10G  0 disk  
├─vda1                      252:1    0  488M  0 part  /boot
└─vda2                      252:2    0  9.5G  0 part  
  ├─centos_softraid-root    253:0    0  8.6G  0 lvm   /
  └─centos_softraid-swap    253:1    0  976M  0 lvm   [SWAP]
vdb                         252:16   0   10G  0 disk  
└─vdb1                      252:17   0   10G  0 part  
  └─md0                       9:0    0   20G  0 raid0 
    └─imlala--vg-imlala--lv 253:2    0   20G  0 lvm   /imlala-data
vdc                         252:32   0   10G  0 disk  
└─vdc1                      252:33   0   10G  0 part  
  └─md0                       9:0    0   20G  0 raid0 
    └─imlala--vg-imlala--lv 253:2    0   20G  0 lvm   /imlala-data
vdd                         252:48   0   10G  0 disk

和之前一样，先创建RAID分区：

fdisk /dev/vdd

这次的流程就可以简化一些了，我们已经知道fd是创建RAID分区：

Command (m for help): n
Partition type:
   p   primary (0 primary, 0 extended, 4 free)
   e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-20971519, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519): 
Using default value 20971519
Partition 1 of type Linux and of size 10 GiB is set

Command (m for help): t
Selected partition 1
Hex code (type L to list all codes): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

分好区后，就可以使用下面的命令将新硬盘添加到阵列了：

mdadm --grow /dev/md0 --raid-devices=3 --add /dev/vdd1

此时我们查看阵列状态可以看到等级变成了RAID4：

/dev/md0:
           Version : 1.2
     Creation Time : Sat Jun 22 22:43:47 2019
        Raid Level : raid4
        Array Size : 20951040 (19.98 GiB 21.45 GB)
     Used Dev Size : 10475520 (9.99 GiB 10.73 GB)
      Raid Devices : 4
     Total Devices : 3
       Persistence : Superblock is persistent

       Update Time : Sun Jun 23 12:05:56 2019
             State : clean, FAILED, reshaping 
    Active Devices : 2
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 1

        Chunk Size : 512K

Consistency Policy : resync

    Reshape Status : 46% complete
     Delta Devices : 1, (3->4)

              Name : softraid:0  (local to host softraid)
              UUID : aba3285e:6ecbfafd:df3daac6:712acf24
            Events : 29

    Number   Major   Minor   RaidDevice State
       0     252       17        0      active sync   /dev/vdb1
       1     252       33        1      active sync   /dev/vdc1
       3     252       49        2      spare rebuilding   /dev/vdd1
       -       0        0        3      removed

这个过程是自动的，一旦转换完成，终端shell会回显一条提示：

mdadm: level of /dev/md0 changed to raid0

此时再次查看阵列状态，可以看到阵列已经回到RAID0了，并且新增加的硬盘也正常工作了：

/dev/md0:
           Version : 1.2
     Creation Time : Sat Jun 22 22:43:47 2019
        Raid Level : raid0
        Array Size : 31426560 (29.97 GiB 32.18 GB)
      Raid Devices : 3
     Total Devices : 3
       Persistence : Superblock is persistent

       Update Time : Sun Jun 23 12:06:27 2019
             State : clean 
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0

        Chunk Size : 512K

Consistency Policy : none

              Name : softraid:0  (local to host softraid)
              UUID : aba3285e:6ecbfafd:df3daac6:712acf24
            Events : 35

    Number   Major   Minor   RaidDevice State
       0     252       17        0      active sync   /dev/vdb1
       1     252       33        1      active sync   /dev/vdc1
       3     252       49        2      active sync   /dev/vdd1

但这里有一个天坑，如果你是使用的LVM，那么此时你会发现LVM找不到PV/VG/LV了，重启服务器也没有用。。

正确的解决办法是重启lvm2-lvmetad服务：

systemctl restart lvm2-lvmetad

然后再扫描一遍PV：

pvscan

此时查看PV：

pvdisplay

丢失的PV就回来了：

--- Physical volume ---
PV Name               /dev/md0
VG Name               imlala-vg
PV Size               19.98 GiB / not usable 4.00 MiB
Allocatable           yes (but full)
PE Size               4.00 MiB
Total PE              5114
Free PE               0
Allocated PE          5114
PV UUID               TfcZTx-MGry-GKL1-osbq-17D7-jGn2-rstT99

但容量怎么样啊？只有20G对不对，现在RAID阵列中新加了一块硬盘，PV的容量应该是30G才对，所以调整PV的大小：

pvresize /dev/md0

再次查看PV的容量，可以看到现在是30G了：

--- Physical volume ---
PV Name               /dev/md0
VG Name               imlala-vg
PV Size               <29.97 GiB / not usable 0   
Allocatable           yes 
PE Size               4.00 MiB
Total PE              7672
Free PE               2558
Allocated PE          5114
PV UUID               TfcZTx-MGry-GKL1-osbq-17D7-jGn2-rstT99

PV的容量增加了，VG卷组的容量也会跟着增加，查看：

vgdisplay

可以看到VG现在也是30G的总容量，并且有10G的FREE：

--- Volume group ---
VG Name               imlala-vg
System ID             
Format                lvm2
Metadata Areas        1
Metadata Sequence No  3
VG Access             read/write
VG Status             resizable
MAX LV                0
Cur LV                1
Open LV               1
Max PV                0
Cur PV                1
Act PV                1
VG Size               <29.97 GiB
PE Size               4.00 MiB
Total PE              7672
Alloc PE / Size       5114 / <19.98 GiB
Free  PE / Size       2558 / 9.99 GiB
VG UUID               Fgsvw8-4TFt-p4WX-w9ig-jstp-K048-DPo68M

但LV呢？LV逻辑卷的容量是不会增加的，查看：

lvdisplay

验证一下，可以看到还是20G的大小：

--- Logical volume ---
LV Path                /dev/imlala-vg/imlala-lv
LV Name                imlala-lv
VG Name                imlala-vg
LV UUID                h5iMee-a49a-7len-V5dJ-XDky-qY2K-dqPSfN
LV Write Access        read/write
LV Creation host, time softraid, 2019-06-22 23:17:01 +0800
LV Status              available
# open                 1
LV Size                <19.98 GiB
Current LE             5114
Segments               1
Allocation             inherit
Read ahead sectors     auto
- currently set to     8192
Block device           253:2

现在我们只需要把LV逻辑卷的容量进行增加即可：

lvresize -rl +100%FREE /dev/imlala-vg/imlala-lv

查看当前的挂载状态，为验证这个容量是完全可用的，我往里面写了29G的数据，可以看到是完全正常的：

[root@softraid imlala-data]# df -hT
Filesystem                        Type      Size  Used Avail Use% Mounted on
/dev/mapper/centos_softraid-root  xfs       8.6G  1.2G  7.4G  14% /
devtmpfs                          devtmpfs  3.9G     0  3.9G   0% /dev
tmpfs                             tmpfs     3.9G     0  3.9G   0% /dev/shm
tmpfs                             tmpfs     3.9G  8.6M  3.9G   1% /run
tmpfs                             tmpfs     3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/vda1                         xfs       485M  181M  304M  38% /boot
tmpfs                             tmpfs     783M     0  783M   0% /run/user/0
/dev/mapper/imlala--vg-imlala--lv xfs        30G   29G  2.0G  94% /imlala-data

RAID0没有容灾机制，所以这篇文章就水到这里了。。