zvol: Enable zvol threading functionality on FreeBSD

[fuporovvStack]

Motivation and Context

Make zvol I/O requests processing asynchronous on FreeBSD side. Clone zvol threading logic and required module parameters from Linux side. Make zvol threadpool creation/destruction logic shared for both Linux and FreeBSD. Use OS physio routines in async mode, in case if zvol is exported in cdev mode. Disable volthreading zfs parameter on FreeBSD side by default.

Description

The reason of making volthreading=off by default on FreeBSD side is the poor performance for single zvol fio tests with voltreading=on comparing with normal mode. The problem lays on FreeBSD side.
The FreeBSD does not supports async IO correctly for zvols or any other char devices. We solved this problem for zvols in volmode=dev by modifying FreeBSD AIO logic under aio_qbio() function. It works synchronously, as could be seen from here.
But it is possible to add additional aio_queue function pointer to struct cdevsw, like it doing in struct fileops for example, and pass struct kaiocb AIO objects directly to zvol, where it will be processed asynchronously. In this case the viewable AIO requests performance increasing could be achieved.
If this PR will be accepted, I am ready to start work on modifications for FreeBSD.

How Has This Been Tested?

The tests/functional/zvol/zvol_stress/zvol_stress.ksh testcase was modified to have volthreading on FreeBSD side.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Performance enhancement (non-breaking change which improves efficiency)
• Code cleanup (non-breaking change which makes code smaller or more readable)
• Breaking change (fix or feature that would cause existing functionality to change)
• Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
• Documentation (a change to man pages or other documentation)

Checklist:

• My code follows the OpenZFS code style requirements.
• I have updated the documentation accordingly.
• I have read the contributing document.
• I have added tests to cover my changes.
• I have run the ZFS Test Suite with this change applied.
• All commit messages are properly formatted and contain Signed-off-by.

[fuporovvStack]

@amotin, @markjdb

[amotin]

I am happy to see somebody working on unifying this code. But see some comments inline:

[fuporovvStack]

I found 4 ways to call zvol, and decided to use next sync/async switching logic dependent of volmode parameter:
geom:
- (1) zvol_geom_bio_start() - sync
- (2) zvol_geom_worker() - async
cdev:
- (3) zvol_cdev_read()/_write() - sync
- (4) zvol_geom_bio_strategy() - async

(1) - the way dependent from geom thread stack usage, case when direct=1.
(2) - case when direct variable value become zero. The direct value is not depend from sync or aio usage.
(3) - normal usage in cdev mode, example: fio with engine=sync.
(4) - used in case of AIO, example: fio with engine=posixaio

[amotin]

(1) - the way dependent from geom thread [stack usage](https://github.com/freebsd/freebsd-src/blob/main/sys/geom/geom_io.c#L566), case when direct=1.

Aside of stack usage, some GEOM classes may be unable to use direct I/O due to locking constraints and have to switch into GEOM up/down threads.