Improve the handling and validation of JSON input/output user settings

[florisvdh]

Description of the requested feature

This relates to the use_json_input and use_json_output user settings through options and environment variables (see ?qgis_using_json_input). And about controlling how input to and output from qgis_process are interfaced: either using JSON or using the legacy (no-JSON) method.

Current behaviour

Currently we have this behaviour:

• with no options set, will favour JSON input and JSON output whenever possible. Essential indeed.
• do I use JSON output? (= outcome of qgis_using_json_output()) With query = TRUE (as implemented by qgis_configure()) takes on the use_json_output user setting if present (if absent try to use JSON output); with query = FALSE (default) returns the cached use_json_output value – the use_json_output user setting is neglected. The cached use_json_output value is only updated when qgis_configure() is run (i.e. reconfigure the package).
• do I use JSON input? (= outcome of qgis_using_json_input()) This is determined by minimum required QGIS version (3.23.0) and JSON output, but if the json_input user setting is set, that setting is taken for granted.
• qgis_using_json_output() and the cached use_json_output value are ignored by qgis_run_algorithm() when qgis_using_json_input() is TRUE, making qgis_run_algorithm() behave in harmony with qgis_process. But this leaves us with inconsistency wrt the return value of qgis_using_json_output() and the cached use_json_output value: these can be FALSE even while qgis_run_algorithm() is implementing TRUE when the json_input user setting is TRUE.
• qgis_using_json_input() always queries the user settings, while qgis_using_json_output() doesn't unless it gets the argument query = TRUE.
• while the use_json_input user setting has immediate effect, both before or after loading the package, the use_json_output user setting does not affect the cached use_json_output value, also not when set before loading the package. Only actively calling qgis_configure() will do that, since the latter uses qgis_using_json_output(query = TRUE).

Note that caching use_json_output makes sense since using JSON output (i.e. calling qgis_process with --json) determines the form and richness of data returned from qgis_algorithms(), which is cached too. Also the value of qgis_plugins() may be different, notably the row order.

Future behaviour?

I believe the following changes or additions will make behaviour more consistent and intuitive:

• consistently make sure, regardless of the user settings and cached use_json_output value, that qgis_using_json_output() cannot return FALSE when qgis_using_json_input() returns TRUE. Given that these functions can take into account user settings, this also means not accepting the corresponding conflicting user setting (or cache value) combination.
  • if qgis_using_json_input() returns TRUE, then qgis_using_json_output() must be TRUE. Equivalent to: if qgis_using_json_output() returns FALSE, then qgis_using_json_input() must be FALSE.
• if the json_input user setting is TRUE, it should be verified that this request can be granted by the backend.
• if the json_input user setting is TRUE and if it can be granted, it should determine the outcome of qgis_using_json_input(), qgis_using_json_output() and the cached use_json_output value: they should all become TRUE.
• just like qgis_using_json_input() instantly queries (and validates) the use_json_input setting and immediately affects package behaviour, qgis_using_json_output() should instantly query and validate the use_json_output setting (validation also taking into account the use_json_input setting), update the cached use_json_output value to its outcome and immediately affect package behaviour; no qgis_configure() needed. Only when the use_json_output setting is missing and query = FALSE, return the cached value instead.
• qgis_configure() is still needed to persistently store the cached use_json_output value and make sure it is aligned with the return value of qgis_algorithms().
• take into account the presence of JSON input/output user settings when validating the cache file during package loading, and reconfigure the package (i.e. including the cache file) if the outcome of those user settings conflicts with the cached use_json_output value. In other words, include this in the checks run by qgis_configure(use_cached_data = TRUE).

I think I have implemented all of this in the json branch. Unit tests are still to be added; so for now still to be regarded as experimental (at least R CMD check is still OK on my machine). Changes should not affect typical use of the package, but only make the experience more intuitive when one wants to fiddle with JSON input/output.

Feel free to check yourself or discuss.

Additional context

Some of these ideas grew out of exploring #185.