Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use CreateFileMapping\MapViewOfFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx on Windows #135

Merged
merged 10 commits into from
Jan 30, 2025

Conversation

simongdavies
Copy link
Contributor

This PR replaces the use of VirtualAllocEx and VirtualFreeEx with CreateFileMapping, MapViewOfFile, UnmapViewOfFile and CloseHandle on Windows.

This enables the mapping of memory into the surrogate processes without zero copying.

There are 9 commits as part of this PR each with its own description/explanation.

@simongdavies simongdavies added the kind/enhancement For PRs adding features, improving functionality, docs, tests, etc. label Jan 16, 2025
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 7 out of 10 changed files in this pull request and generated no comments.

Files not reviewed (3)
  • docs/assets/hyperlight_arch.excalidraw: Language not supported
  • src/hyperlight_host/src/hypervisor/hyperv_windows.rs: Evaluated as low risk
  • src/hyperlight_host/src/hypervisor/hypervisor_handler.rs: Evaluated as low risk
Comments suppressed due to low confidence (3)

src/hyperlight_host/src/mem/shared_mem.rs:111

  • The new behavior introduced in the Drop implementation of HostMapping should be covered by tests.
if let Err(e) = unsafe { UnmapViewOfFile(mem_mapped_address) } {

src/hyperlight_host/src/hypervisor/surrogate_process.rs:85

  • There is a spelling mistake in the error message: 'drropping' should be 'dropping'.
tracing::error!("Failed to get surrogate process manager when drropping SurrogateProcess: {:?}", e);

src/hyperlight_host/src/hypervisor/surrogate_process.rs:60

  • [nitpick] The variable 'mem_mapped_address' should follow Rust's naming convention and be renamed to 'memory_mapped_view_address'.
let mem_mapped_address = MEMORY_MAPPED_VIEW_ADDRESS {

@simongdavies simongdavies force-pushed the use-mmap-files branch 3 times, most recently from d28e54c to 0c9a093 Compare January 16, 2025 22:12
@simongdavies simongdavies requested a review from Copilot January 16, 2025 22:13
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 7 out of 10 changed files in this pull request and generated no comments.

Files not reviewed (3)
  • docs/assets/hyperlight_arch.excalidraw: Language not supported
  • src/hyperlight_host/src/hypervisor/hyperv_windows.rs: Evaluated as low risk
  • src/hyperlight_host/src/mem/mgr.rs: Evaluated as low risk
Comments suppressed due to low confidence (4)

src/hyperlight_host/src/sandbox/uninitialized.rs:1162

  • The error message should be clear and consistent. Consider rephrasing to: 'GuestBinary not found: 'some/path/that/does/not/exist': The system cannot find the path specified (os error)'.
matches!(sbox, Err(e) if e.to_string().contains("GuestBinary not found: 'some/path/that/does/not/exist': The system cannot find the path specified. (os error"))

src/hyperlight_host/src/hypervisor/surrogate_process.rs:66

  • The error message should be updated to reflect the new function name UnmapViewOfFile2.
Failed to free surrogate process resources (UnmapViewOfFile2 failed): {:?}

src/hyperlight_host/src/hypervisor/surrogate_process.rs:85

  • The error message should be updated to reflect the new function name return_surrogate_process.
Failed to get surrogate process manager when dropping SurrogateProcess: {:?}

src/hyperlight_host/src/hypervisor/surrogate_process.rs:64

  • Ensure that the new behavior introduced by UnmapViewOfFile2 is adequately covered by tests.
if let Err(e) = unsafe { UnmapViewOfFile2(process_handle, memory_mapped_view_address, flags) } {

@simongdavies simongdavies force-pushed the use-mmap-files branch 3 times, most recently from 0e2c611 to 7f56eec Compare January 16, 2025 23:15
@simongdavies simongdavies changed the title Use CreateFileMapping\MapViewOFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx` on Windows Use CreateFileMapping\MapViewOfFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx` on Windows Jan 16, 2025
marosset
marosset previously approved these changes Jan 17, 2025
Copy link
Contributor

@marosset marosset left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@simongdavies simongdavies changed the title Use CreateFileMapping\MapViewOfFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx` on Windows Use CreateFileMapping\MapViewOfFile and UnmapViewOfFile\CloseHandle instead of VitualAllocEx and VirtualFreeEx on Windows Jan 21, 2025
ludfjig
ludfjig previously approved these changes Jan 27, 2025
Copy link
Contributor

@ludfjig ludfjig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@simongdavies simongdavies dismissed stale reviews from ludfjig and marosset via e5ec0a4 January 28, 2025 21:59
@simongdavies simongdavies force-pushed the use-mmap-files branch 2 times, most recently from 850ffb2 to 08e57ea Compare January 28, 2025 23:02
ludfjig
ludfjig previously approved these changes Jan 28, 2025
Copy link
Contributor

@ludfjig ludfjig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Regarding the HANDLE insteade of HandleWrapper, it just seemed unnecessary to have send/sync on handle when it wasn't needed, but if it's indeed in needed, or that made things more complicated feel free to change it back

syntactically
syntactically previously approved these changes Jan 28, 2025
@simongdavies
Copy link
Contributor Author

LGTM. Regarding the HANDLE insteade of HandleWrapper, it just seemed unnecessary to have send/sync on handle when it wasn't needed, but if it's indeed in needed, or that made things more complicated feel free to change it back

It is needed in some places but not in the one where you pointed it out, since I have to make another small change I will check and see if there are other places where it can be removed.

@simongdavies simongdavies dismissed stale reviews from syntactically and ludfjig via 60d9a17 January 29, 2025 19:47
@simongdavies simongdavies force-pushed the use-mmap-files branch 2 times, most recently from 60d9a17 to a61ab99 Compare January 29, 2025 19:49
syntactically
syntactically previously approved these changes Jan 29, 2025
ludfjig
ludfjig previously approved these changes Jan 29, 2025
Reflect the use of CreateFileMapping/MapViewOfFile/UnmapViewOfFile/CloseHandle instead of VirtualAlloc/VirtualFree

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
…owsDriver

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
When the driver is created a handle to the FileMapping for the memory for this sandbox.

Uses that handle as an argument to the get_surrogate_process function on the SurrogateProcessManager

Removes the code that copies the Sandbox memory to and from the surrogate process memory each time the VM is run.

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Replace use of VirtualAlloc/VirtualFree with CreateFileMapping/MapViewOFile/UnmapViewOfFile/CloseHandle to allow memry to be mapped into surrogate processes without any copying

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
… feature flag

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
…tualFreeEx

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
…ualAllocEx

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
…m_SystemServices

Signed-off-by: Simon Davies <simongdavies@users.noreply.github.com>
@simongdavies simongdavies merged commit 203215b into hyperlight-dev:main Jan 30, 2025
21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement For PRs adding features, improving functionality, docs, tests, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants