Virtual Data Optimizer (VDO)

 ( blackluck | 2018. május 14., hétfő - 10:27 )

Ha mar kijott a centos 7.5 es abban is benne van vdo futolag raneztem mit is tud.

Rovid infonak: Red Hat kivezette btrfs-t, zfs-t nem akarja/tudja (license vagy egyeb gondok miatt) bevezetni, igy inkabb hibrid megoldas mellett tette le a voksat software storage megoldasok fele. Ehhez deduplikaciot es tomoritest blokk szinten ezzel a 7.5-os rhel-ben megjelent vdo-val tervezi megvalositani.
Blokkszinten tortenik, lehet filerendszer alatt, volume group alatt stb. is hasznalni, filrendszertol fuggetlenul, van par megkotese azert, pl thin provison lvm-re nem lehet tenni es tarsai.

Egyszeruseg kedveert egy lvm volume-ot (main-storage) hasznaltam fel erre, mert szabad fizikai diszk nem volt:

# fdisk -l /dev/mapper/main-storage

Disk /dev/mapper/main-storage: 2788.9 GB, 2788935335936 bytes, 5447139328 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 262144 bytes / 1310720 bytes

Parameterezes kell neki erosen, default parameterekkel legtobb dologra 1 thread-et hasznal es egy filerendszer letrehozas is hosszu percekig tartott neki ugy (oke 2.5T tarhely, de akkor is), gepben 6mag+HT proci van, az alapjan nagyjabol hasrautesszeruen adtam csak be lenti parametereket, nyilvan lehet jobban is, de csak gyors tesztre ez mar hasznalhato volt.
Parameterezesek jo reszet letrehozaskor lehet allitani, masik reszet akkor is lehet ha csak leallitjuk vdo eszkozt miutan nem hasznalja semmi (pl: vdo stop stor) es nem kell ujraformazni a tarhelyet.

# vdo create -n stor --device /dev/mapper/main-storage --vdoLogicalSize=5T --vdoSlabSize=32G --sparseIndex=enabled --indexMem=0.25 --vdoCpuThreads=6 --vdoLogicalThreads=3 --vdoHashZoneThreads=3--vdoHashZoneThreads=3
Creating VDO stor
Starting VDO stor
Starting compression on VDO stor

Ezzel letrejon egy ujabb mapper device a megadott 5T merettel. Ezt a meretet ha nem adjuk meg akkor ugyanannyi lesz mint a valos tarhely. Novelni elvileg lehet kesobb, persze akkor a rajta levo filerendszert vagy egyebet is utana kell allitani. Ez az ami olyan hogy meg kell probalni megsaccolni milyen adatot fogunk rajta tarolni az mennyire lesz tomoritheto, hogy mennyivel lohetjuk fole a virtualis tarhelyet a valosnak (nem neztem mit kezd vele ha elfogyott a valos hely).

# fdisk -l /dev/mapper/stor

Disk /dev/mapper/stor: 5497.6 GB, 5497558138880 bytes, 1342177280 sectors
Units = sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

filerendszer letrehozas:

# mkfs.xfs -K /dev/mapper/stor
meta-data=/dev/mapper/stor isize=512 agcount=5, agsize=268435455 blks
= sectsz=4096 attr=2, projid32bit=1
= crc=1 finobt=0, sparse=0
data = bsize=4096 blocks=1342177275, imaxpct=5
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal log bsize=4096 blocks=521728, version=2
= sectsz=4096 sunit=1 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0

Vdo status lekerdezes:

# vdostats --human-readable
Device Size Used Available Use% Space saving%
/dev/mapper/stor 2.5T 6.1G 2.5T 0% 99%

Ahogy latszik hogy alapbol foglal helyet a deduplikacios indexeknek, meg lehet egyebeknek is hala, de aranyaiban nem veszes.

Reszletesebb statusz:

# vdo status
VDO status:
Date: '2018-05-11 13:49:27+02:00'
Node: test-stor.pelda.hu
Kernel module:
Loaded: true
Name: kvdo
Version information:
kvdo version: 6.1.0.153
Configuration:
File: /etc/vdoconf.yml
Last modified: '2018-05-11 13:48:45'
VDOs:
stor:
Acknowledgement threads: 1
Activate: enabled
Bio rotation interval: 64
Bio submission threads: 4
Block map cache size: 128M
Block map period: 16380
Block size: 4096
CPU-work threads: 6
Compression: enabled
Configured write policy: auto
Deduplication: enabled
Device mapper status: 0 10737418240 vdo /dev/dm-4 albserver online cpu=6,bio=4,ack=1,bioRotationInterval=64
Emulate 512 byte: disabled
Hash zone threads: 3
Index checkpoint frequency: 0
Index memory setting: 0.25
Index parallel factor: 0
Index sparse: enabled
Index status: online
Logical size: 5T
Logical threads: 3
Physical size: 2659736M
Physical threads: 1
Read cache: disabled
Read cache size: 0M
Slab size: 32G
Storage device: /dev/mapper/main-storage
VDO statistics:
/dev/mapper/stor:
1K-blocks: 2723569664
1K-blocks available: 2717158776
1K-blocks used: 6410888
512 byte emulation: false
KVDO module bios used: 37412
KVDO module bytes used: 1148231168
KVDO module peak bio count: 37700
KVDO module peak bytes used: 1152119776
bios acknowledged discard: 0
bios acknowledged flush: 1
bios acknowledged fua: 0
bios acknowledged partial discard: 0
bios acknowledged partial flush: 0
bios acknowledged partial fua: 0
bios acknowledged partial read: 0
bios acknowledged partial write: 0
bios acknowledged read: 782
bios acknowledged write: 521840
bios in discard: 0
bios in flush: 1
bios in fua: 0
bios in partial discard: 0
bios in partial flush: 0
bios in partial fua: 0
bios in partial read: 0
bios in partial write: 0
bios in progress discard: 0
bios in progress flush: 0
bios in progress fua: 0
bios in progress read: 0
bios in progress write: 0
bios in read: 782
bios in write: 521840
bios journal completed discard: 0
bios journal completed flush: 0
bios journal completed fua: 0
bios journal completed read: 0
bios journal completed write: 76568
bios journal discard: 0
bios journal flush: 76568
bios journal fua: 76568
bios journal read: 0
bios journal write: 76568
bios meta completed discard: 0
bios meta completed flush: 0
bios meta completed fua: 0
bios meta completed read: 709
bios meta completed write: 76635
bios meta discard: 0
bios meta flush: 76569
bios meta fua: 76569
bios meta read: 709
bios meta write: 76635
bios out completed discard: 0
bios out completed flush: 0
bios out completed fua: 0
bios out completed read: 4
bios out completed write: 46
bios out discard: 0
bios out flush: 0
bios out fua: 0
bios out read: 4
bios out write: 46
bios page cache completed discard: 0
bios page cache completed flush: 0
bios page cache completed fua: 0
bios page cache completed read: 648
bios page cache completed write: 0
bios page cache discard: 0
bios page cache flush: 0
bios page cache fua: 0
bios page cache read: 648
bios page cache write: 0
block map cache pressure: 0
block map cache size: 134217728
block map clean pages: 0
block map dirty pages: 648
block map discard required: 0
block map failed pages: 0
block map failed reads: 0
block map failed writes: 0
block map fetch required: 648
block map flush count: 0
block map found in cache: 1028594
block map free pages: 32118
block map incoming pages: 0
block map outgoing pages: 0
block map pages loaded: 648
block map pages saved: 0
block map read count: 522087
block map read outgoing: 0
block map reclaimed: 0
block map wait for page: 14715
block map write count: 521870
block size: 4096
completed recovery count: 0
compressed blocks written: 2
compressed fragments in packer: 12
compressed fragments written: 28
current VDO IO requests in progress: 12
current dedupe queries: 0
data blocks used: 16
dedupe advice stale: 0
dedupe advice timeouts: 0
dedupe advice valid: 1
entries indexed: 43
flush out: 1
instance: 1
invalid advice PBN count: 0
journal blocks batching: 0
journal blocks committed: 76568
journal blocks started: 76568
journal blocks writing: 0
journal blocks written: 76568
journal commits requested count: 0
journal disk full count: 0
journal entries batching: 0
journal entries committed: 1044573
journal entries started: 1044573
journal entries writing: 0
journal entries written: 1044573
logical blocks: 1342177280
logical blocks used: 521820
maximum VDO IO requests in progress: 2000
maximum dedupe queries: 13
no space error count: 0
operating mode: normal
overhead blocks used: 1602706
physical blocks: 680892416
posts found: 1
posts not found: 43
queries found: 0
queries not found: 0
read cache accesses: 0
read cache data hits: 0
read cache hits: 0
read only error count: 0
read-only recovery count: 0
recovery progress (%): N/A
reference blocks written: 0
release version: 131337
saving percent: 99
slab count: 81
slab journal blocked count: 0
slab journal blocks written: 0
slab journal disk full count: 0
slab journal flush count: 0
slab journal tail busy count: 0
slab summary blocks written: 0
slabs opened: 1
slabs reopened: 0
updates found: 28
updates not found: 0
used percent: 0
version: 26
write amplification ratio: 0.15
write policy: sync

Ahogy latszik dedup es tomorites is be van kapcsolva, ez a default.

Elkezdtem belemasolni tobbnyire csv/tsv file-okat, ezek jo resze mar alapbol tomoritve volt gzip-el. Ahogy sejtheto a tomoritett file-okon mar nemigazan tud nyerni, par szazalekot csak kb, a tomoritettleneken viszont 70-80%-ot is hozta akar. Kivancsisagbol letoltottem par linux telepito iso dvd-t is, azokon is csak par szazalekot tudott nyerni.
Persze kozben terhelte a procit erezhetoen. Meg latszik hogy parameterezesnel kellene allitani ha jobban teljesitmeny szamitana, mert nem egyenletesen oszlik el igy se, jo esellyel memoria ertekeket kellene novelni, hogy nagyobb legyen a buffer amivel dolgozhat.

Valogatas nelkul ment ra a masolas tomoritett es tomoritettlen file-okkal, aranyaiban vegen joval tobb volt a tomoritett, igy 30% koruli nyereseg lett csak belole:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/stor 5.0T 3.5T 1.6T 70% /home/storage
Device Size Used Available Use% Space saving%
/dev/mapper/stor 2.5T 2.4T 131.5G 94% 30%

De korabban mikor meg jobban voltak tomoritettlen file-ok aranyaiban tobbet is tudott:
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/stor 5.0T 293G 4.8T 6% /home/storage
Device Size Used Available Use% Space saving%
/dev/mapper/stor 2.5T 109.7G 2.4T 4% 68%

Osszesegeben csodat ez se tud (egyelore), alap beallitasai eleg szegenyesek es bizony be kell alligatni (bar gondolom idovel ez javulni fog hogy automatan megtegye majd a fontosabb ertekeket), illetve a tarhely meretet is a felhasznalt adatokhoz saccolva kell igazitani. Meg persze terheli is a gepet es lassitja is emiatt a diszk muveleteket amivel szamolni kell.