2021-06-30 linux学习-硬盘的高级操作(一)VDO虚拟数据优化

  • Post author:
  • Post category:linux



Virtual Data Optimize

是一种通过压缩或删除存储设备上的数据来优化存储空间的技术,简称VDO。

最常见的就是

去重

功能,大大节省了硬盘空间。接下来,我们就来实践一下!

专家建议做虚拟机或

容器

时,采用逻辑与物理

10:1

的比率进行配置,即使用1TB物理存储对应10TB的逻辑存储;而做

对象存储

时(例如Ceph)则采用

3:1

的比率进行配置,即使用1TB物理存储对应3TB的逻辑存储。


VDO虚拟数据优化

一 创建VDO卷

1 首先在虚拟机里添加l一块10G硬盘,进过查询硬盘为sdb,这里不做演示。

2 创建VDO卷,卷名xiaofeng 容量100G

vdo create –name=xiaofeng –device=/dev/sdb –vdoLogicalSize=100G

[root@linuxprobe ~]# vdo create --name=xiaofeng --device=/dev/sdb --vdoLogicalSize=100G
Creating VDO xiaofeng
Starting VDO xiaofeng
Starting compression on VDO xiaofeng
VDO instance 0 volume is ready at /dev/mapper/xiaofeng

二 查看刚才创建的VDO卷信息 vdo status –name=xiaofeng

[root@linuxprobe ~]# vdo status --name=xiaofeng
VDO status:
  Date: '2021-06-30 08:13:53+08:00'
  Node: linuxprobe.com
Kernel module:
  Loaded: true
  Name: kvdo
  Version information:
    kvdo version: 6.2.0.293
Configuration:
  File: /etc/vdoconf.yml
  Last modified: '2021-06-30 08:07:51'
VDOs:
  xiaofeng:
    Acknowledgement threads: 1
    Activate: enabled
    Bio rotation interval: 64
    Bio submission threads: 4
    Block map cache size: 128M
    Block map period: 16380
    Block size: 4096
    CPU-work threads: 2
    Compression: enabled
    Configured write policy: auto
    Deduplication: enabled
    Device mapper status: 0 209715200 vdo /dev/sdb normal - online online 1049638 2621440
    Emulate 512 byte: disabled
    Hash zone threads: 1
    Index checkpoint frequency: 0
    Index memory setting: 0.25
    Index parallel factor: 0
    Index sparse: disabled
    Index status: online
    Logical size: 100G
    Logical threads: 1
    Max discard size: 4K
    Physical size: 10G
    Physical threads: 1
    Slab size: 2G
    Storage device: /dev/disk/by-id/ata-VMware_Virtual_SATA_Hard_Drive_01000000000000000001
    VDO statistics:
      /dev/mapper/xiaofeng:
        1K-blocks: 10485760
        1K-blocks available: 6287208
        1K-blocks used: 4198552
        512 byte emulation: false
        KVDO module bios used: 37286
        KVDO module bytes used: 424949088
        KVDO module peak bio count: 37574
        KVDO module peak bytes used: 424952976
        bios acknowledged discard: 0
        bios acknowledged flush: 0
        bios acknowledged fua: 0
        bios acknowledged partial discard: 0
        bios acknowledged partial flush: 0
        bios acknowledged partial fua: 0
        bios acknowledged partial read: 0
        bios acknowledged partial write: 0
        bios acknowledged read: 261
        bios acknowledged write: 0
        bios in discard: 0
        bios in flush: 0
        bios in fua: 0
        bios in partial discard: 0
        bios in partial flush: 0
        bios in partial fua: 0
        bios in partial read: 0
        bios in partial write: 0
        bios in progress discard: 0
        bios in progress flush: 0
        bios in progress fua: 0
        bios in progress read: 0
        bios in progress write: 0
        bios in read: 261
        bios in write: 0
        bios journal completed discard: 0
        bios journal completed flush: 0
        bios journal completed fua: 0
        bios journal completed read: 0
        bios journal completed write: 0
        bios journal discard: 0
        bios journal flush: 0
        bios journal fua: 0
        bios journal read: 0
        bios journal write: 0
        bios meta completed discard: 0
        bios meta completed flush: 0
        bios meta completed fua: 0
        bios meta completed read: 4
        bios meta completed write: 65
        bios meta discard: 0
        bios meta flush: 1
        bios meta fua: 1
        bios meta read: 4
        bios meta write: 65
        bios out completed discard: 0
        bios out completed flush: 0
        bios out completed fua: 0
        bios out completed read: 0
        bios out completed write: 0
        bios out discard: 0
        bios out flush: 0
        bios out fua: 0
        bios out read: 0
        bios out write: 0
        bios page cache completed discard: 0
        bios page cache completed flush: 0
        bios page cache completed fua: 0
        bios page cache completed read: 0
        bios page cache completed write: 0
        bios page cache discard: 0
        bios page cache flush: 0
        bios page cache fua: 0
        bios page cache read: 0
        bios page cache write: 0
        block map cache pressure: 0
        block map cache size: 134217728
        block map clean pages: 0
        block map dirty pages: 0
        block map discard required: 0
        block map failed pages: 0
        block map failed reads: 0
        block map failed writes: 0
        block map fetch required: 0
        block map flush count: 0
        block map found in cache: 0
        block map free pages: 32768
        block map incoming pages: 0
        block map outgoing pages: 0
        block map pages loaded: 0
        block map pages saved: 0
        block map read count: 0
        block map read outgoing: 0
        block map reclaimed: 0
        block map wait for page: 0
        block map write count: 0
        block size: 4096
        completed recovery count: 0
        compressed blocks written: 0
        compressed fragments in packer: 0
        compressed fragments written: 0
        concurrent data matches: 0
        concurrent hash collisions: 0
        current VDO IO requests in progress: 0
        current dedupe queries: 0
        data blocks used: 0
        dedupe advice stale: 0
        dedupe advice timeouts: 0
        dedupe advice valid: 0
        entries indexed: 0
        flush out: 0
        instance: 0
        invalid advice PBN count: 0
        journal blocks batching: 0
        journal blocks committed: 0
        journal blocks started: 0
        journal blocks writing: 0
        journal blocks written: 0
        journal commits requested count: 0
        journal disk full count: 0
        journal entries batching: 0
        journal entries committed: 0
        journal entries started: 0
        journal entries writing: 0
        journal entries written: 0
        logical blocks: 26214400
        logical blocks used: 0
        maximum VDO IO requests in progress: 59
        maximum dedupe queries: 0
        no space error count: 0
        operating mode: normal
        overhead blocks used: 1049638
        physical blocks: 2621440
        posts found: 0
        posts not found: 0
        queries found: 0
        queries not found: 0
        read only error count: 0
        read-only recovery count: 0
        recovery progress (%): 0
        reference blocks written: 0
        release version: 133524
        saving percent: 0
        slab count: 3
        slab journal blocked count: 0
        slab journal blocks written: 0
        slab journal disk full count: 0
        slab journal flush count: 0
        slab journal tail busy count: 0
        slab summary blocks written: 0
        slabs opened: 0
        slabs reopened: 0
        updates found: 0
        updates not found: 0
        used percent: 40
        version: 30
        write amplification ratio: 0.0
        write policy: sync

输出信息中包含了VDO卷创建的时间、主机名、版本、是否删重(Deduplication)及是否压缩(Compression)等关键指标。

三 格式化VDO卷

1 格式化VDO卷 mkfs.xfs /dev/mapper/xiaofeng

[root@linuxprobe ~]# mkfs.xfs /dev/mapper/xiaofeng 
meta-data=/dev/mapper/xiaofeng   isize=512    agcount=4, agsize=6553600 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1
data     =                       bsize=4096   blocks=26214400, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=12800, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

2 刷新内核信息以确保同步 udevadm settle

[root@linuxprobe ~]# udevadm settle

四 挂载VDO卷

1 新建一个用于挂载的目录 mkdir -p /root/new_disk/vdo/xiaofeng

[root@linuxprobe ~]# mkdir -p /root/new_disk/vdo/xiaofeng

2 挂载VDO卷 mount /dev/mapper/xiaofeng /root/new_disk/vdo/xiaofeng

[root@linuxprobe ~]# mount /dev/mapper/xiaofeng /root/new_disk/vdo/xiaofeng

3 查看VOD卷的真实大小  vdostats –human-readable

[root@linuxprobe ~]# vdostats --human-readable
Device                    Size      Used Available Use% Space saving%
/dev/mapper/xiaofeng     10.0G      4.0G      6.0G  40%           99%

4 查看VDO卷的逻辑大小 df -hT

[root@linuxprobe ~]# df -hT
Filesystem            Type      Size  Used Avail Use% Mounted on
devtmpfs              devtmpfs  3.9G     0  3.9G   0% /dev
tmpfs                 tmpfs     3.9G     0  3.9G   0% /dev/shm
tmpfs                 tmpfs     3.9G  9.9M  3.9G   1% /run
tmpfs                 tmpfs     3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/mapper/rhel-root xfs        44G  4.1G   40G  10% /
/dev/sda1             xfs      1014M  152M  863M  15% /boot
tmpfs                 tmpfs     796M   16K  796M   1% /run/user/42
tmpfs                 tmpfs     796M  3.5M  792M   1% /run/user/0
/dev/mapper/xiaofeng  xfs       100G  747M  100G   1% /root/new_disk/vdo/xiaofeng

五 开机自动挂载

1 查看VDO卷的UUID    blkid /dev/mapper/xiaofeng

[root@linuxprobe ~]# blkid /dev/mapper/xiaofeng 
/dev/mapper/xiaofeng: UUID="6e1d7281-c010-4b8b-9044-b157d09a8bbd" TYPE="xfs"

2 编辑/etc/fstab  vim /etc/fstab

[root@linuxprobe ~]# vim /etc/fstab

在最后加上一行

UUID=6e1d7281-c010-4b8b-9044-b157d09a8bbd   /root/new_disk/vdo/xiaofeng      xfs   defaults,_netdev   0 0


#
# /etc/fstab
# Created by anaconda on Tue Jun  8 06:02:20 2021
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/rhel-root   /                       xfs     defaults     0 0
UUID=2c1949d3-5cd8-4b71-b60d-8885bfc326e0 /boot                   xfs     defaults  0 0
/dev/mapper/rhel-swap   swap                    swap    defaults        0 0
UUID=6e1d7281-c010-4b8b-9044-b157d09a8bbd   /root/new_disk/vdo/xiaofeng      xfs   defaults,_netdev   0 0
~    

3 挂载所有/etc/fstab中的设备 mount -a

[root@linuxprobe ~]# mount -a

4 重启系统 reboot

[root@linuxprobe ~]# reboot

5 查看是否挂载成功  vdostats –human-readable

[root@linuxprobe ~]# vdostats --human-readable
Device                    Size      Used Available Use% Space saving%
/dev/mapper/xiaofeng     10.0G      4.0G      6.0G  40%           99%

6 查看是否挂载成功   df -hT

[root@linuxprobe ~]# df -hT
Filesystem            Type      Size  Used Avail Use% Mounted on
devtmpfs              devtmpfs  3.9G     0  3.9G   0% /dev
tmpfs                 tmpfs     3.9G     0  3.9G   0% /dev/shm
tmpfs                 tmpfs     3.9G  9.9M  3.9G   1% /run
tmpfs                 tmpfs     3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/mapper/rhel-root xfs        44G  4.1G   40G  10% /
/dev/sda1             xfs      1014M  152M  863M  15% /boot
/dev/mapper/xiaofeng  xfs       100G  747M  100G   1% /root/new_disk/vdo/xiaofeng
tmpfs                 tmpfs     796M   16K  796M   1% /run/user/42
tmpfs                 tmpfs     796M  3.5M  792M   1% /run/user/0

发现已经有挂载信息,大功告成!



版权声明:本文为x629242原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。