Re: Pozor na harddisky s nedokumentovaným použitím SMR

[Petr Labaj]

Tak jsem zkusil to "fio" na obou discích. Vždy na první i druhé 
partition. První má 100 GB, druhá 900 GB.
Obě jsou plné skoro kompletně plné (cca 10 GB volného místa na každé).
Výsledky sem dávám jen v syrovém stavu.
Na těch discích jsou filesystémy z Windows, ale test probíhal na Linuxu. 
Takže na ne-nativním filesystému.
Při spuštění na obou discích vyhlásil chybu "fio: native_fallocate call 
failed: Operation not supported", ale pokračoval.

Příkaz a vypsaná hlavička:
fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 
--name=test --filename=random_read_write.fio --bs=4k --iodepth=64 
--size=4G --readwrite=randrw --rwmixread=75 --runtime=60 --time_based
test: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 
4096B-4096B, ioengine=libaio, iodepth=64
fio-3.1
Starting 1 process
test: Laying out IO file (1 file / 4096MiB)
fio: native_fallocate call failed: Operation not supported

Výsledky:

********* Disk Seagate ST2000DM008 se SMR, systémová partiton 100GB:

Jobs: 1 (f=1): [m(1)][100.0%][r=608KiB/s,w=196KiB/s][r=152,w=49 
IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=3375: Fri Jun  5 04:07:43 2020
    read: IOPS=150, BW=602KiB/s (617kB/s)(35.4MiB/60174msec)
    bw (  KiB/s): min=  200, max=  800, per=99.59%, avg=599.56, 
stdev=85.90, samples=120
    iops        : min=   50, max=  198, avg=149.88, stdev=21.44, samples=120
   write: IOPS=50, BW=202KiB/s (207kB/s)(11.9MiB/60174msec)
    bw (  KiB/s): min=   88, max=  328, per=99.83%, avg=200.67, 
stdev=45.37, samples=120
    iops        : min=   22, max=   82, avg=50.17, stdev=11.34, samples=120
   cpu          : usr=0.37%, sys=1.18%, ctx=15306, majf=0, minf=9
   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.3%, 
 >=64=99.5%
      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, 
 >=64=0.0%
      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, 
 >=64=0.0%
      issued rwt: total=9059,3035,0, short=0,0,0, dropped=0,0,0
      latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
    READ: bw=602KiB/s (617kB/s), 602KiB/s-602KiB/s (617kB/s-617kB/s), 
io=35.4MiB (37.1MB), run=60174-60174msec
   WRITE: bw=202KiB/s (207kB/s), 202KiB/s-202KiB/s (207kB/s-207kB/s), 
io=11.9MiB (12.4MB), run=60174-60174msec

Disk stats (read/write):
   sdb: ios=7281/2531, merge=0/3, ticks=57024/1566828, in_queue=1653208, 
util=95.67%

********* Disk Seagate ST2000DM008 se SMR, datová partiton 900GB:

Jobs: 1 (f=1): [m(1)][100.0%][r=728KiB/s,w=232KiB/s][r=182,w=58 
IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=3389: Fri Jun  5 04:13:27 2020
    read: IOPS=179, BW=718KiB/s (735kB/s)(42.2MiB/60145msec)
    bw (  KiB/s): min=  440, max=  904, per=99.54%, avg=714.68, 
stdev=81.01, samples=120
    iops        : min=  110, max=  226, avg=178.67, stdev=20.25, samples=120
   write: IOPS=59, BW=239KiB/s (245kB/s)(14.1MiB/60145msec)
    bw (  KiB/s): min=  128, max=  416, per=99.78%, avg=238.47, 
stdev=51.91, samples=120
    iops        : min=   32, max=  104, avg=59.62, stdev=12.98, samples=120
   cpu          : usr=0.43%, sys=1.37%, ctx=18161, majf=0, minf=10
   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.2%, 
 >=64=99.6%
      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, 
 >=64=0.0%
      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, 
 >=64=0.0%
      issued rwt: total=10797,3601,0, short=0,0,0, dropped=0,0,0
      latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
    READ: bw=718KiB/s (735kB/s), 718KiB/s-718KiB/s (735kB/s-735kB/s), 
io=42.2MiB (44.2MB), run=60145-60145msec
   WRITE: bw=239KiB/s (245kB/s), 239KiB/s-239KiB/s (245kB/s-245kB/s), 
io=14.1MiB (14.7MB), run=60145-60145msec

Disk stats (read/write):
   sdb: ios=8451/3428, merge=0/3425, ticks=56680/1623808, 
in_queue=1680472, util=95.31%

********* Starší serverový disk Seagate 2 TB, systémová partiton 100GB:

Jobs: 1 (f=1): [m(1)][100.0%][r=340KiB/s,w=120KiB/s][r=85,w=30 IOPS][eta 
00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=3549: Fri Jun  5 04:33:07 2020
    read: IOPS=178, BW=712KiB/s (729kB/s)(41.0MiB/60358msec)
    bw (  KiB/s): min=  152, max=  992, per=99.91%, avg=711.34, 
stdev=201.12, samples=120
    iops        : min=   38, max=  248, avg=177.83, stdev=50.28, samples=120
   write: IOPS=59, BW=238KiB/s (243kB/s)(14.0MiB/60358msec)
    bw (  KiB/s): min=   24, max=  416, per=100.00%, avg=237.33, 
stdev=79.32, samples=120
    iops        : min=    6, max=  104, avg=59.33, stdev=19.83, samples=120
   cpu          : usr=0.42%, sys=1.39%, ctx=18053, majf=0, minf=9
   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.2%, 
 >=64=99.6%
      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, 
 >=64=0.0%
      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, 
 >=64=0.0%
      issued rwt: total=10748,3586,0, short=0,0,0, dropped=0,0,0
      latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
    READ: bw=712KiB/s (729kB/s), 712KiB/s-712KiB/s (729kB/s-729kB/s), 
io=41.0MiB (44.0MB), run=60358-60358msec
   WRITE: bw=238KiB/s (243kB/s), 238KiB/s-238KiB/s (243kB/s-243kB/s), 
io=14.0MiB (14.7MB), run=60358-60358msec

Disk stats (read/write):
   sdb: ios=8023/2452, merge=0/1, ticks=56788/1394340, in_queue=1514216, 
util=94.81%

********* Starší serverový disk Seagate 2 TB, datová partiton 900GB:

Jobs: 1 (f=1): [m(1)][100.0%][r=860KiB/s,w=252KiB/s][r=215,w=63 
IOPS][eta 00m:00s]
test: (groupid=0, jobs=1): err= 0: pid=3530: Fri Jun  5 04:29:34 2020
    read: IOPS=171, BW=687KiB/s (703kB/s)(40.3MiB/60059msec)
    bw (  KiB/s): min=  216, max= 1032, per=99.70%, avg=683.97, 
stdev=221.82, samples=120
    iops        : min=   54, max=  258, avg=170.98, stdev=55.45, samples=120
   write: IOPS=57, BW=230KiB/s (235kB/s)(13.5MiB/60059msec)
    bw (  KiB/s): min=   40, max=  424, per=99.62%, avg=228.13, 
stdev=88.96, samples=120
    iops        : min=   10, max=  106, avg=57.03, stdev=22.24, samples=120
   cpu          : usr=0.33%, sys=1.41%, ctx=17351, majf=0, minf=7
   IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.1%, 32=0.2%, 
 >=64=99.5%
      submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, 
 >=64=0.0%
      complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.1%, 
 >=64=0.0%
      issued rwt: total=10311,3447,0, short=0,0,0, dropped=0,0,0
      latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
    READ: bw=687KiB/s (703kB/s), 687KiB/s-687KiB/s (703kB/s-703kB/s), 
io=40.3MiB (42.2MB), run=60059-60059msec
   WRITE: bw=230KiB/s (235kB/s), 230KiB/s-230KiB/s (235kB/s-235kB/s), 
io=13.5MiB (14.1MB), run=60059-60059msec

Disk stats (read/write):
   sdb: ios=7741/2689, merge=0/2691, ticks=56992/1838580, 
in_queue=1895556, util=95.40%

*********************

Dne 4.6.2020 v 16:22 Josef Dvoracek napsal(a):
> jen bych poznamenal, že dle mého porozumění problematiky je SMR spíše 
> háklivé na náhodný přístup na disk.., když jde přístup na tzv. šindele 
> sekvenčně za sebou, tak by ke zpomalení dojít nemělo.
>
> Pokud máte chuť, tady je nějaká cmdline benchmarku, kterým jsem kdysi 
> zkoušel nějaký svůj cluster, fio je standardně dostupné ve všech 
> běžných distribucích:
>
> fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 
> --name=test --filename=random_read_write.fio --bs=4k --iodepth=64 
> --size=4G --readwrite=randrw --rwmixread=75 --runtime=60 --time_based
>
> bylo by zajímavé, pokud byste výše zmíněnou cli spustil v nějakém 
> prázdném adresáři obou disků..Test vždy poběží 60 sec a uvidíte, kolik 
> práce stihnul..
> Není potřeba root, zabere 4G místa.
>
> JD