feat: LHv2: add note about non-NVMe disk size requirements

This one is a little obscure.  When testing with non-NVMe disks,
Longhorn uses the aio bdev driver, which assumes the block size
is 4096 bytes, and thus requires the disk to be an even multiple
of that figure.  We hit this in testing as described in
harvester/harvester#6861.

Ordinarily one should be using NVMe disks, which end up using the
nvme bdev driver, which picks up the actual block size of the
underlying device and everything just works.  This subtlety
(nvme bdev driver working regardless of disk size, but aio bdev
driver requiring 4096 byte blocks) is apparently not documented
anywhere else (e.g. it's not mentioned in the Longhorn docs)
which is why I'm adding it here, just in case anyone else hits
it and gets confused.

Signed-off-by: Tim Serong <tserong@suse.com>

docs/advanced/longhorn-v2.md

@@ -71,6 +71,11 @@ The Longhorn V2 Data Engine is only available for newly created volumes. Existin
 
   Set the `Provisioner` of each extra disk to `Longhorn V2 (CSI)`.
 
+  :::note
+
+  Harvester sets the Longhorn disk driver to "auto" as described in [the Longhorn documentation](https://longhorn.io/docs/1.7.2/v2-data-engine/features/node-disk-support/). This means that NVMe disks will use the nvme bdev driver which provides the best performance. If testing with non-NVMe disks (e.g. SSDs), note that these will fall back to using the aio bdev driver, which requires that the disk size be an even multiple of 4096 bytes. Non-NVMe disks that do not meet this size constraint will not be added successfully.
+  :::
+
 1. Go to **Advanced** > **Storage Classes**, and then add a new StorageClass as described in [Creating a StorageClass](storageclass.md#creating-a-storageclass). 
 
   Set the `Provisioner` to `Longhorn V2 (CSI)`.