Merge master into update guidance improvement feature branch #5416
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In CA-314001 VM import was running fsync, which caused all other threads in XAPI to be blocked, which in turn caused the HA xapi health checker to fence the host. We should release the runtime lock whenever we perform a syscall, especially those that perform I/O, since they can block for a long time (minutes) if there are a lot of dirty pages and the storage is slow. The change here follows the recommendations from the manual section 20.12.2 Parallel execution of long-running C code http://caml.inria.fr/pub/docs/manual-ocaml/intfc.html Signed-off-by: Edwin Török <edvin.torok@citrix.com>
CA-314001: release runtime lock around long running system calls
Json so far is generated in a naive way that violates the Json
specification by ignoring basic quoting rules. This commit delegates
generation of Json to a library instead and should be much more
declarative.
However, the generated JSON still is not equivalent to the XML as it
contains hard-coded values in some places where the XML seems to hold
actual data. As can be seen from the code,
* last_ds:0.0,
* value:0.0,
* unknown_sec:0
are hard coded in the Json output but not in the XML output.
{
"name": "memory_total_kib",
"type": "GAUGE",
"minimal_hearbeat": "300.0000",
"min": "0.0",
"max": "Infinity",
"last_ds": "0.00",
"value": "0.00",
"unknown_sec": "0.00"
},
<ds>
<name>memory_total_kib</name>
<type>GAUGE</type>
<minimal_heartbeat>300.0000</minimal_heartbeat>
<min>0.0</min>
<max>Infinity</max>
<last_ds>33548544</last_ds>
<value>160004678.0795</value>
<unknown_sec>0</unknown_sec>
</ds>
Signed-off-by: Christian Lindig <christian.lindig@citrix.com>
Some entries in the Json output where hard-coded compared to the XML output. This commit adds these data point to the Json output. Signed-off-by: Christian Lindig <christian.lindig@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Also convert to use alcotest's infrastructure so checks are more verbose when failing. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Also move tests from build to build-tests so the get run by travis script. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Data Sources with derive type are not monotonically increasing like counter ones and may lead to strange metrics being produced. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Previously the values were just clamped before being applied the transform function from the client. Now the values marked as unknown values after having the transform applied, in case the transforms do not check for out-of-bounds values. Treating out-of-bounds values as Unknown matches rrdtool's behaviour. Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
CA-322008: Stop treating Derive datasources as counters
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
tests: add json marshalling
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Test array length invariant in the rrds Test how rrd_updates are populated (depends a lot on current implementation) Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
Signed-off-by: Pau Ruiz Safont <pau.safont@citrix.com>
…rdk/python-add-code-coverage-reporting Major Python CI update: - Cancel obsolete CI jobs from the same pull request or same branch with pushing a branch. - Convert GitHub CI from using docker to LizardByte/setup-python-action for Python 2.7 - Add an initial pre-commit configuration file for pre-commit and pre-push test hooks. - Add running the pre-commit configuration as part of the GitHub CI Python action workflow - Enable a GitHub ProblemMatcher for converting Python warnings into GitHub annotations - Add reviewdog/action-actionlint@v1 to lint GitHub action workflow files. - Add an inital pyproject.toml file with default configuraitons for isort, black, mypy, pytype and pytype_reporter.py (see below for more about it) - Add https://github.com/reviewdog/action-pyflakes to error on unused variables and unused imports if they are in the context of the submitted change. - Add collection of code coverage from the pytest run in GitHub CI and integrate Codecov.io. - Add dciborow/action-pylint@0.1.0 (configured for reviewdog review comments on "**/*.py": - Its level is only warning, so it will never make the build fail because of pylint warnings. - It will only provide review comments (if adding review comments is permitted) - Add pytype_reporter.py: Can run local and in GitHub CI to get pytype reports. - If the action has write permission to add PR comments, it will at a status comment! - It is designed to report the pytype status on the GitHub Action Summary page - It is designed to fail only when a changed file is either included in regular tests or is changed but still in the expected_to_fail list: Any changed file should be fully fixed for pytype (using ignores if needed) and removed from the expected_to_fail list. - Remove the python shebang from scripts/examples/python/XenAPIPlugin.py because it is not an executable script but a boilerplate module for XenAPI python plugins. - Also fix a pylint warning about its non-conformant module name, add a docstring and a link to the documentation for using it. Guideline: In case any of the new checkers cause any sort of problem for you, please feel free to disable or remove the one checker that you run into. In general, it should be easy to find the documentation for one checker that issues reports and either fix them if there is an issue or silence/disable this specific check, preferably (if feasible) only at the few locations that it may report. This is a big departure from having only pytest in GitHub CI to having multiple checks. I certainly hope that all goes well and that we manage the transition well. The best way to work with this (and with pull request reviews in general) is to only change one python script per pull request to keep the pull request focused on one topic! I wish you good checks and that the code quality may improve as a result!
…2059 CA-372059: refactor the type of host in `squeeze.ml`
This now generates a TabError. Signed-off-by: Yann Dirson <yann.dirson@vates.fr>
This uses the same mechanism as ac683ca, to deal with occurrence of `long` causing `make test` to fail with python3, but also occurrences of `unicode`. Signed-off-by: Yann Dirson <yann.dirson@vates.fr>
This uses the same mechanism as ac683ca. Signed-off-by: Yann Dirson <yann.dirson@vates.fr>
With this a "make test" after build out of OPAM on Debian 12 finishes successfully. Signed-off-by: Yann Dirson <yann.dirson@vates.fr>
Reported-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com> Signed-off-by: Yann Dirson <yann.dirson@vates.fr>
Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
Moves `module Component` in a separate file `xapi_observer_components.ml`. Moves `module EnvRecord` in a seperate file `xapi_aux/env_record.ml` Basic funcionality from `Xapi_observer` cannot be used from `sm_exec.ml` because cycle dependencies. This commit does some housekeeping and splits `xapi_observer.ml` into `xapi_observer.ml` and `xapi_observer_components.ml` while also moving `module EnvRecord` under `xapi_aux`. `xapi_observer_components.ml` also contains the functions that act on components independent of the observers. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Adds a `xapi_observer_component.ml` function `is_component_enabled` -> `component:t` --> `bool`. Returns `true` if the given `component` is enabled in any of the observers. A helper function is needed to check if `SMapi` component is enabled in any of the observers. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
…_exec` calls Adds `Tracing.SpanContext.trace_id_of_span_context` as an env variable. Adds a new entry `observer_config_dir` in `xapi_globs.ml`. Sets this new entry as an env variable. Currently there is no path for SMAPI `observer.py` calls to access their respective `TRACEPARENT`, nor to access the configuration files for observers. This commit sets: - `TRACEPARENT` as an environment variable if any smapi observer component is enabled; - `OBSERVER_CONFIG_DIR` as an environment variable if any smapi observer component is enabled. This contains the path to the enabled observers configuration files. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Defines `default_path_env_pair` as the default value used by optional parameter `?env`. External clients of `Forkhelpers` need to include the value `default_path_env_pair` when passing an argument to `?env`. Now this value is defined inside the module for client use. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Moves `config_root` under `observer_confi_dir` in `xapi_globs.ml`. Adds `observer_conf_path` function that constructs the path based on `uuid`. Removes `~config_root` parameter. There were various ways to define paths is `Dom0ObserverConfig` which increase the probabilty of introducing bugs in the future. This consolidates them into `observer_conf_path_of`. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
Splits functionality `env_vars_of_observer` into `traceparent_of_dbg` under `debuginfo.ml` and `env_vars_of_component` under `xapi_observer_components.ml. Doing the processing of traceparent and of the necessary observer env vars under their respective modules increases the readablity of `sm_exec.ml`. Signed-off-by: Gabriel Buica <danutgabriel.buica@cloud.com>
…/CP-46377 CP-46377: Add env vars `TRACEPARENT` & `OBSERVER_CONFIG_DIR` to `exec_xmlrpc` calls
Otherwise dune runs them for all packages in the repository, failing if they don't depend on the xapi-rrd library. Signed-off-by: Pau Ruiz Safont <pau.ruizsafont@cloud.com>
This will be useful for checking some new fields introduced in Xen-4.15+ Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Only `platform:nested-virt=true` sets this field. The previous experimental field `platform:exp-nested-hvm=true` is now dropped. `platform:nested-virt` has been present in the product since Dundee, any users of `exp-nested-hvm` can be updated to use that instead. This doesn't mean that nested virt is supported yet: its implementation is still incomplete. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
`platform:msr-relaxed=true` Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Edwin Török <edwin.torok@cloud.com>
`platform:vpmu=true`. For this to work the Xen cmdline also needs to be edited to remove the watchdog, and enable vpmu. In the future this may become easier. VPMU doesn't require HVM. Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Set to 2 if the host supports it. Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Always set to 0. Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Signed-off-by: Edwin Török <edwin.torok@cloud.com>
Merge Xen-4.17 update
Signed-off-by: Gang Ji <gang.ji@citrix.com>
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## feature/update-guidance-improvement #5416 +/- ##
======================================================================
Coverage ? 45.38%
======================================================================
Files ? 18
Lines ? 2937
Branches ? 0
======================================================================
Hits ? 1333
Misses ? 1604
Partials ? 0
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
Drop it as there is one to merge latest master: #5425 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Below is the change (in the merging commit: 7d0f375) when merging from master:
[gangj@xenrt1015818055 scm]$ git show 7d0f375
commit 7d0f375 (HEAD -> private/gangj/ugi.a.merge_master.2, my_gh/private/gangj/merge_from_master)
Merge: 2ce256e fe5d334
Author: Gang Ji gang.ji@citrix.com
Date: Fri Feb 2 15:12:08 2024 +0800
diff --cc ocaml/idl/schematest.ml
index 7f029c22d,cb84842f7..b3f86ab2d
--- a/ocaml/idl/schematest.ml
+++ b/ocaml/idl/schematest.ml
@@@ -3,7 -3,7 +3,7 @@@ let hash x = Digest.string x |> Digest.
(* BEWARE: if this changes, check that schema has been bumped accordingly in
ocaml/idl/datamodel_common.ml, usually schema_minor_vsn *)
- let last_known_schema_hash = "49c51c0b5e895f42a1c9c19abb038b41"
-let last_known_schema_hash = "1e43ef93af9de55620fda75281e8a992"
++let last_known_schema_hash = "5b597309a69b2c7b9053446fa338c701"
let current_schema_hash : string =
let open Datamodel_types in
diff --cc quality-gate.sh
index bf8f561af,224e852aa..d33edacff
--- a/quality-gate.sh
+++ b/quality-gate.sh
@@@ -3,7 -3,7 +3,7 @@@
set -e
list-hd () {
- N=315
- N=318
++ N=316
LIST_HD=$(git grep -r --count 'List.hd' -- **/*.ml | cut -d ':' -f 2 | paste -sd+ - | bc)
if [ "$LIST_HD" -eq "$N" ]; then
echo "OK counted $LIST_HD List.hd usages"