Tags: firecracker-microvm/firecracker
Tags
Added * [#4346](#4346): Added support to emit aggregate (minimum/maximum/sum) latency for `VcpuExit::MmioRead`, `VcpuExit::MmioWrite`, `VcpuExit::IoIn` and `VcpuExit::IoOut`. The average for these VM exits is not emitted since it can be deduced from the available emitted metrics. * [#4360](#4360): Added dev-preview support for backing a VM's guest memory by 2M hugetlbfs pages. Please see the [documentation](docs/hugepages.md) for more information * [#4490](#4490): Added block and net device metrics for file/tap access latencies and queue backlog lengths, which can be used to analyse saturation of the Firecracker VMM thread and underlying layers. Queue backlog length metrics are flushed periodically. They can be used to esimtate an average queue length by request by dividing its value by the number of requests served. Changed * [#4230](#4230): Changed microVM snapshot format version strategy. Firecracker snapshot format now has a version that is independent of Firecracker version. The current version of the snapshot format is v1.0.0. From now on, the Firecracker binary will define the snapshot format version it supports and it will only be able to load snapshots with format that is backwards compatible with that version. Users can pass the `--snapshot-version` flag to the Firecracker binary to see its supported snapshot version format. This change renders all previous Firecracker snapshots (up to Firecracker version v1.6.0) incompatible with the current Firecracker version. * [#4449](#4449): Added information about page size to the payload Firecracker sends to the UFFD handler. Each memory region object now contains a `page_size_kib` field. See also the [hugepages documentation](docs/hugepages.md). * [#4501](#4501): Only use memfd to back guest memory if a vhost-user-blk device is configured, otherwise use anonymous private memory. This is because serving page faults of shared memory used by memfd is slower and may impact workloads. Fixed * [#4409](#4409): Fixed a bug in the cpu-template-helper that made it panic during conversion of cpu configuration with SVE registers to the cpu template on aarch64 platform. Now cpu-template-helper will print warnings if it encounters SVE registers during the conversion process. This is because cpu templates are limited to only modify registers less than 128 bits. * [#4413](#4413): Fixed a bug in the Firecracker that prevented it to restore snapshots of VMs that had SVE enabled. * [#4414](#4360): Made `PATCH` requests to the `/machine-config` endpoint transactional, meaning Firecracker's configuration will be unchanged if the request returns an error. This fixes a bug where a microVM with incompatible balloon and guest memory size could be booted, due to the check for this condition happening after Firecracker's configuration was updated. * [#4259](#4259): Added a double fork mechanism in the Jailer to avoid setsid() failures occurred while running Jailer as the process group leader. However, this changed the behaviour of Jailer and now the Firecracker process will always have a different PID than the Jailer process. [#4436](#4436): Added a "Known Limitations" section in the Jailer docs to highlight the above change in behaviour introduced in PR#4259. [#4442](#4442): As a solution to the change in behaviour introduced in PR#4259, provided a mechanism to reliably fetch Firecracker PID. With this change, Firecracker process's PID will always be available in the Jailer's root directory regardless of whether new_pid_ns was set. * [#4468](#4468): Fixed a bug where a client would hang or timeout when querying for an MMDS path whose content is empty, because the 'Content-Length' header field was missing in a response.
Added * [#4145](#4145): Added support for per net device metrics. In addition to aggregate metrics `net`, each individual net device will emit metrics under the label `"net_{iface_id}"`. E.g. the associated metrics for the endpoint `"/network-interfaces/eth0"` will be available under `"net_eth0"` in the metrics json object. * [#4202](#4202): Added support for per block device metrics. In addition to aggregate metrics `block`, each individual block device will emit metrics under the label `"block_{drive_id}"`. E.g. the associated metrics for the endpoint `"/drives/{drive_id}"` will be available under `"block_drive_id"` in the metrics json object. * [#4205](#4205): Added a new `vm-state` subcommand to `info-vmstate` command in the `snapshot-editor` tool to print MicrovmState of vmstate snapshot file in a readable format. Also made the `vcpu-states` subcommand available on x86_64. * [#4063](#4063): Added source-level instrumentation based tracing. See [tracing](./docs/tracing.md) for more details. * [#4138](#4138), [#4170](#4170), [#4223](#4223), [#4247](#4247), [#4226](#4226): Added **developer preview only** (NOT for production use) support for vhost-user block devices. Firecracker implements a vhost-user frontend. Users are free to choose from existing open source backend solutions or their own implementation. Known limitation: snapshotting is not currently supported for microVMs containing vhost-user block devices. See the [related doc page](./docs/api_requests/block-vhost-user.md) for details. The device emits metrics under the label `"vhost_user_{device}_{drive_id}"`. Changed * [#4309](#4309): The jailer's option `--parent-cgroup` will move the process to that cgroup if no `cgroup` options are provided. * Simplified and clarified the removal policy of deprecated API elements to follow semantic versioning 2.0.0. For more information, please refer to [this GitHub discussion](#4135). * [#4180](#4180): Refactored error propagation to avoid logging and printing an error on exits with a zero exit code. Now, on successful exit "Firecracker exited successfully" is logged. * [#4194](#4194): Removed support for creating Firecracker snapshots targeting older versions of Firecracker. With this change, running 'firecracker --version' will not print the supported snapshot versions. * [#4301](#4301): Allow merging of diff snapshots into base snapshots by directly writing the diff snapshot on top of the base snapshot's memory file. This can be done by setting the `mem_file_path` to the path of the pre-existing full snapshot. Deprecated * [#4209](#4209): `rebase-snap` tool is now deprecated. Users should use `snapshot-editor` for rebasing diff snapshots. Fixed * [#4171](#4171): Fixed a bug that ignored the `--show-log-origin` option, preventing it from printing the source code file of the log messages. * [#4178](#4178): Fixed a bug reporting a non-zero exit code on successful shutdown when starting Firecracker with `--no-api`. * [#4261](#4261): Fixed a bug where Firecracker would log "RunWithApiError error: MicroVMStopped without an error: GenericError" when exiting after encountering an emulation error. It now correctly prints "RunWithApiError error: MicroVMStopped *with* an error: GenericError". * [#4242](#4242): Fixed a bug introduced in #4047 that limited the `--level` option of logger to Pascal-cased values (e.g. accepting "Info", but not "info"). It now ignores case again. * [#4286](#4286): Fixed a bug in the asynchronous virtio-block engine that rendered the device non-functional after a PATCH request was issued to Firecracker for updating the path to the host-side backing file of the device. * [#4301](#4301): Fixed a bug where if Firecracker was instructed to take a snapshot of a microvm which itself was restored from a snapshot, specifying `mem_file_path` to be the path of the memory file from which the microvm was restored would result in both the microvm and the snapshot being corrupted. It now instead performs a "write-back" of all memory that was updated since the snapshot was originally loaded.
Added * [#4287](#4287) Document a caveat to the jailer docs when using the `--parent-cgroup` option, which results in it being ignored by the jailer. Refer to the [jailer documentation](./docs/jailer.md#caveats) for a workaround. Changed * [#4191](#4191): Refactored error propagation to avoid logging and printing an error on exits with a zero exit code. Now, on successful exit "Firecracker exited successfully" is logged. Fixed * [#4277](#4277): Fixed a bug that ignored the `--show-log-origin` option, preventing it from printing the source code file of the log messages. * [#4179](#4179): Fixed a bug reporting a non-zero exit code on successful shutdown when starting Firecracker with `--no-api`. * [#4271](#4271): Fixed a bug where Firecracker would log "RunWithApiError error: MicroVMStopped without an error: GenericError" when exiting after encountering an emulation error. It now correctly prints "RunWithApiError error: MicroVMStopped *with* an error: GenericError". * [#4270](#4270): Fixed a bug introduced in #4047 that limited the `--level` option of logger to Pascal-cased values (e.g. accepting "Info", but not "info"). It now ignores case again. * [#4295](#4295): Fixed a bug in the asynchronous virtio-block engine that rendered the device non-functional after a PATCH request was issued to Firecracker for updating the path to the host-side backing file of the device.
Fixed * Fixed a change in behavior of normalize host brand string that breaks Firecracker on external instances. * Fixed the T2A CPU template not to unset the MMX bit (CPUID.80000001h:EDX[23]) and the FXSR bit (CPUID.80000001h:EDX[24]). * Fixed the T2A CPU template to set the RstrFpErrPtrs bit (CPUID.80000008h:EBX[2]).
Added * Added support for custom CPU templates allowing users to adjust vCPU features exposed to the guest via CPUID, MSRs and ARM registers. * Introduced V1N1 static CPU template for ARM to represent Neoverse V1 CPU as Neoverse N1. * Added support for the `virtio-rng` entropy device. The device is optional. A single device can be enabled per VM using the `/entropy` endpoint. * Added a `cpu-template-helper` tool for assisting with creating and managing custom CPU templates. Changed * Set FDP_EXCPTN_ONLY bit (CPUID.7h.0:EBX[6]) and ZERO_FCS_FDS bit (CPUID.7h.0:EBX[13]) in Intel's CPUID normalization process. Fixed * Fixed feature flags in T2S CPU template on Intel Ice Lake. * Fixed CPUID leaf 0xb to be exposed to guests running on AMD host. * Fixed a performance regression in the jailer logic for closing open file descriptors. Related to: [#3542](#3542). * A race condition that has been identified between the API thread and the VMM thread due to a misconfiguration of the `api_event_fd`. * Fixed CPUID leaf 0x1 to disable perfmon and debug feature on x86 host. * Fixed passing through cache information from host in CPUID leaf 0x80000006. * Fixed the T2S CPU template to set the RRSBA bit of the IA32_ARCH_CAPABILITIES MSR to 1 in accordance with an Intel microcode update. * Fixed the T2CL CPU template to pass through the RSBA and RRSBA bits of the IA32_ARCH_CAPABILITIES MSR from the host in accordance with an Intel microcode update. * Fixed passing through cache information from host in CPUID leaf 0x80000005. * Fixed the T2A CPU template to disable SVM (nested virtualization). * Fixed the T2A CPU template to set EferLmsleUnsupported bit (CPUID.80000008h:EBX[20]), which indicates that EFER[LMSLE] is not supported.