I was thinking about putting U.2 disks in my workstation to get optimum speed. However, reading about the subject, I found a few gotchas:
The U.2 disks will not work in future U.3 slots, but U.3 drives will work on U.2 controllers, according to this article. That is despite all the effort for interoperability promised by tri-mode HBAs. The implication is that it is better to avoid U.2 now and wait for more U.3 products in the months ahead.
Supermicro's HBA list
does not include any PCIe 4.0 product now, using only 50% of the speed of the slots in Talos II. The PCIe 4.0 cards appear to be arriving in the market very slowly.
It seems that the PCIe 4.0 products that do exist now are not fully supported yet on every OS, e.g. the LSI 9500-8i
For people running Fedora, they will go with Btrfs as the default filesystem soon. As far as I know, Btrfs mirrored configurations require two copies of each write to go through the bus from CPU to HBA, compared to mirroring in hardware RAID, where the OS only has to send a single write to the RAID controller. This appears to be another good reason to have the fastest possible storage path, PCIe 4.0
The conclusion: I can live with SATA SSDs running on my LSI 9207-4i4e for 6-12 months while the U.3 drives and PCIe 4.0 Tri-mode controllers become more certain.
