Author Topic: ZFS performance on Blackbird  (Read 32 times)

alci

  • Newbie
  • *
  • Posts: 1
  • Karma: +0/-0
    • View Profile
ZFS performance on Blackbird
« on: May 19, 2020, 10:30:10 am »
Hi,

I'm trying to get decent performance with ZFS on my Blackbird system, running Ubuntu 20.04 with 128 GB RAM. So far, I failed...

I have tried to make things as simple as possible, and using a sinngle SSD (Samsung 840 Pro, 512 GB), tested with bonnie++, here is what I get :

franck@blackbird:~$ bonnie++ -d data -c 8 -m zfs-sdb -s 270000 -r 130688
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.98       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Name:Size etc        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
zfs-sdb  270000M::8   85k  99  475m  89  152m  47  185k  99  499m  53  4070 115
Latency             98772us    3795us   18291us   50503us   16191us    7025us
Version  1.98       ------Sequential Create------ --------Random Create--------
zfs-sdc             -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 685584420  99 +++++ +++ 1143148205  14 -1321910944  98 +++++ +++ -382954332  99
Latency              4631us    2108us    5575ms    4596us     140us     320us
1.98,1.98,zfs-sdb,8,1588801810,270000M,,8192,5,85,99,486661,89,155677,47,185,99,510981,53,4070,115,16,,,,,19138,99,+++++,+++,2146,14,19113,98,+++++,+++,14365,99,98772us,3795us,18291us,50503us,16191us,7025us,4631us,2108us,5575ms,4596us,140us,320us

Notice the huge latency (5545 ms) on sequential create / delete, and the poor sequential input block performance.

Under ext4, the same disk gives me:

franck@blackbird:~$ bonnie++ -d data -c 8 -m ext4-sdb -s 270000 -r 130688
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.98       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Name:Size etc        /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
ext4-sdb 270000M::8  331k  99  481m  31  169m  15 1585k  99  465m  12 12131  87
Latency             27081us   71048us   54910us    9400us   23853us    1330us
Version  1.98       ------Sequential Create------ --------Random Create--------
ext4-sdb            -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency               927us     566us     720us     122us       4us      44us
1.98,1.98,ext4-sdb,8,1588605414,270000M,,8192,5,331,99,492198,31,173283,15,1585,99,476038,12,12131,87,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,27081us,71048us,54910us,9400us,23853us,1330us,927us,566us,720us,122us,4us,44us

I also tested it with pgbench, and ZFS gives me 30% less TPS than ext4.

So, here is my question: should I keep fighting, or is ZFS on Linux known to be badly supported/tested on ppc64 ?
Do you have any experience regarding ZFS on Blackbird ?

Thanks in advance,
Franck