Skip to content

Scenario: Windows Terminal 2.0 Process Model Improvements #5000

New issue
New issue
Closed
#14935
Closed
Scenario: Windows Terminal 2.0 Process Model Improvements#5000
#14935
Assignees
zadjii-msft
Labels
Area-UserInterfaceIssues pertaining to the user interface of the Console or TerminalIn-PRThis issue has a related PRIssue-ScenarioNeeds-Tag-FixDoesn't match tag requirementsProduct-TerminalThe new Windows Terminal.
Milestone
 
Terminal v1.18
@zadjii-msft

Description

@zadjii-msft
zadjii-msft
opened on Mar 18, 2020

This scenario is tracking a number of enhancements we'd like to make to the fundamental process model of the Windows Terminal in v2.0. The following deliverables all have something to do with the way the Terminal is launched, or the way connections or instances are managed, and as such, the solution to any one of these areas should make sure to take into account the others.

Window / Session Managment

Elevation

  • Megathread: sudo, runas, mixed elevation of tabs, etc. #1032 Mixed Elevation
    • This is a hard problem, and I'm making no firm commitments that we'll be able to solve it for sure in 2.0.
    • The goal of this deliverable is to find a safe way that we can support both elevated (High-IL) and unelevated (Medium-IL) tabs, panes, etc all in the same window.
    • Ideally, if a user creates a new elevated tab from an unelevated window, then all the unelevated tabs would seamlessly appear in a new elevated HWND hosting both unelevated and elevated tabs simultaneously.
    • Elevated connections can't be hosted in an unelevated HWND, because that allows for a trivial escalation-of-privilege vector utilizing the Terminal.
    • This problem should arguably be investigated first. If this is something that's technically possible, it will certainly have an influence on the way the rest of these deliverables are architected.
    • ❌ Resolution: This is not going to be technically achievable, unfortunately.
  • [Question] Configuring Windows Terminal profile to always launch elevated #632 Open a Profile as elevated
    • If the above is not achievable technically, then the user should still be able to specify a way to force a profile to open elevated.
    • If mixed elevation isn't possible:
      • If the current window isn't currently elevated, open a new window when someone's profile is marked "elevated: true"
      • If the current window is elevated, open in the same window.

Default Terminal

  • Change Windows OS to support default terminal [defterm] #492 Default Terminal Application
    • This one's a little different than the others. While the rest were Windows Terminal specific features, this deliverable is more of a OS feature than a Terminal feature.
    • This concerns the ability for an arbitrary application to be registered with the OS as the "Default Terminal Application", and be started automatically when the user starts an arbitrary commandline application.
    • This also has some interaction with the above deliverables. When a commandline application is started this way, should they open as a new tab in an existing Terminal window, or start a new window?

Specs

TODOs

These are also tracked in https://github.com/microsoft/terminal/projects/5. This provides a nested list, to mentally track which things are dependent upon other things. This is basically a copy of a page of my notebook.

Window Management

Tear Out

Currently planned state, notes:

See #5000 (comment) for the most up-to-date todo list. There were some dramatic changes to our plans in early 2023, which basically entirely obsoleted "Process Model 2" (where each termcontrol is its own process) in favor of "Process Model 3" (where all windows are all one process).

Original tear-out plans
  • wt.exe accepts --content guid on the commandline to start in content mode. Additionally, Make the Control able to have the Core OOP.
    • these two were originally different bullet points, but I think they need to be combined into one singular PR. They don't make sense alone.
    • in content process mode, it registers ContentProcess for that GUID with CoRegisterClassObject
    • ContentProcess::Initialize(IControlSettings, IConnection) instantiates a new ControlInteractivity (in the content process) (if one hasn't already been initialized)
    • ContentProcess::RequestSwapChainHandle(pid) will duplicate the handle if the pid is different
    • wt properly tracks the lifetime of its content. When that's discarded by the last window process, we'll exit.
    • Do the above line, but better. Now we're using a lock and an event, which is ew. Instead, we should do the Conhost ExitThread() thing on the main thread, and then ExitProcess() when the content process is dtor'd.
    • Move all the chaos I've introduced in main.cpp into its own dedicated file/class or something.
    • An embedding process needs a way to know that the content process is ready, better than a Sleep(2000)
      • A handle to an event is probably the cleanest way - have the window create the event, then pass the handle in CreateProcess & on the commandline to wt --content {guid} --signal {handle}, then have the content process set the event when it's ready.
    • 📝 in dev/migrie/oop/the-whole-thing dev/migrie/oop/infinity-war
    • The Connection is made in the core
    • When the Core signals that the swapchain changed, and the TermControl asks what the swapchain handle is, the Control passes in it's PID. If the PID isn't the same, DuplicateHandle the swapchain to that PID.
    • The sample is updated with a version of the Control that's also OOP
    • If the content process dies, the control needs to be able to display a message box.
    • Make sure UIA signaling still works (accevent.exe in the SDK, C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x64)
    • Make sure that reading the contents of the buffer with Accessibility Insights still works
      • 💩
      • InteractivityAutomationPeer extends Windows.UI.Xaml.Automation.Peers.AutomationPeer. But we're making the interactivity peer in the content process, not in the control layer.
      • When we construct the IAP, we throw instantly, because the WUX..AutomationPeer can't be created off the main thread.
      • How do we only expose the WUX..AutomationPeer via TermControl, but not IAP?
    • The _raiseContentDied should display the error internal to the TermControl. Kinda like the renderer warning, but for this, we need to kill the SwapChainPanel too.
    • in dev/migrie/oop/the-whole-thing dev/migrie/oop/infinity-war
    • Relevant diff: 23a19c5...5c17603
    • Add a velocity flag to gate behind dev builds only. Probably safe to just gate the --content flag for now.
    • Guard calls to _interactivity, _core in TermControl. Those objects might go out of scope at any moment, so pretty much all access needs to be try/caught.
      • we may want to do a function-level try, with a CATCH_RAISE_DIALOG() macro we write to raise the connection died notice. Might be the most minimal diff.
      • Could likely be a follow up to original PR.
      • There's a silly idea in dev/migrie/oop/2/cptn-marvel. I really hate it, it's a HUGE amount of churn.
    • There's a lot of debug spew that makes me think the OOP core does something off the main thread.
      • BriAl and I synced - this is nothing to worry about. This is basically just COM logging "i'm about to increment the refcount"
      • Still trying to find a viable way to suppress that logging.
    • What the hell did I do with the automation peer. Was what was in main correct? Revisit the original PRs for inspiration.
    • ControlCore::_IsClosing TODO!
    • ContentProcess::RequestSwapChainHandle tracelogging TODO!
    • TermControlAutomationPeer::GetLocalizedControlTypeCore that string used to come from localized resources, it can't anymore.
    • The Control properly manages the lifetime of the HANDLE duplicated to it. A wil::unique_handle?
    • IMPORTANT: For whatever reason, when the second window attaches to the content, it renders basically nothing. Input gets sent just fine, but the output doesn't render.. Fixed in 7b3ca83
    • 📝 Now moved to dev/migrie/oop/2/infinity-war
    • 👀 in Add support for running the terminal core as a content process #12938
  • TermControls need to have a local TerminalSettings instance. (Related: Make all the ICoreSettings / IControlSettings properties getters-only #7219)
    • What a shockingly simple oversight. We did all this work to get the connections in the same proc as the TermControl, we totally forgot the Settings.
    • Make sure updating/changing settings works fine across processes
    • Make sure previewing the color scheme still works. That works with some parent/child trickery that won't work OOP.
    • 📝 in dev/migrie/oop/ragnarok
  • Some misc cleanup before endgame:
    • existingConnection in NewTab was redundant / unused
    • _SplitPane renamed to _SplitPaneActiveTab etc.
    • Refactoring to allow the TerminalTab to be initialized without a control
    • Other stuff to, to minimize the diff
    • not started, in dev/migrie/oop/2/black-widow cause this one's not really consequential, is it now.
  • Make the app able to do the OOP instantiation.
  • Make the sample able to pop a control into a new window.
    • ✔️ in dev/migrie/oop/the-whole-thing. I didn't pop it into a new window, just made it so multiple windows could connect to the same one.
  • Defterm is out of proc.
    • DEFTERM: Incoming connections need to be tossed to the ContentProcess. Guh.
    • not started in dev/migrie/oop/2/captain-falcon.
  • Tabs can be serialized to a json blob, with their tab & pane structure.
  • move-pane -w <id> to move a pane/tab to another Terminal window. This will prove we can move panes without the whole instantiation of a new window process.
  • When a tab is torn out of a Terminal window, it can be dropped & re-attached onto another terminal window
  • When a tab is torn out of a Terminal window, it can be dropped somewhere else and turned into a new terminal window
    • not started yet, in dev/migrie/oop/no-way-home
  • When a tab is torn out, we'll create a new window for the tabs left behind, and move the current window around.
    • We'll need to still be able to drop this window on an existing window somehow.
    • Use EvanK's prototype here. We're gonna need to extend the drag bar over the TabView and do everything ourselves
    • No padding in tab well to move the window #12616
    • not started yet, in dev/migrie/oop/multiverse-of-madness

other notes

Just moving these lower in the body to make managing this doc easier

This is a pseudo design that Dustin and I discussed. There's a lot more work that needs to be done here, but I need to save this somewhere outside of a Teams chat history 
---
struct TerminalControl: winrt::implements<MUX::IUIElement> {} // XAML Control
// calls: interactivity.PointerMoved(Point{x, y});
// calls: interactivity.PointerClicked(Point{x, y});
// links: TerminalCore.lib <----> or consumes TerminalCore.dll from RT
^^^ TerminalControl.dll ^^^

---
struct WPFTerminalControl { HWND _h; }
// calls: interactivity.PointerMoved(Point{x, y});
// calls: interactivity.PointerClicked(Point{x, y});
// links: TerminalCore.lib
^^^ WPFTerminalControl.dll ^^^

vvv TerminalCore.lib vvv // Cancelled. TerminalCore.lib or .dll
struct Coord {};
struct PointerInfo {};
struct KeyInfo {};
struct TerminalControlInteractivity;
struct ControlCore; // < implements ... something? // this should just be TerminalCore?
struct TerminalCore; // < implements the "conhosty APIs (ITerminalDispatch, etc.)
// links: dx.lib buffer.lib types.lib

vvv TerminalCore.UnitTests.dll vvv
// contains: interactivity tests
// contains: core tests
// links: TerminalCore.lib
  • Split TerminalControl into lib and dll
  • Split TermControl into Control.TermControl and Control.ControlCore and Control.Interactivity
  • Add some Core.ControlCore tests from things in Tests we should maybe write one day #7001
  • Create a WinRT boundary on Core.Interactivity between it and TermControl
  • Enable Core.ControlCore to exist out-of-proc with XAML, or in-proc
    • This might involve creating a scratch project to consume an in-proc one and an out-of-proc one
  • the Core can create connections, or have them passed in. (so an OOP core can have a connection in it's own process)
  • Much more. This is just what's immediately on my radar.
_4-30-21_: I need to do the DX `HANDLE` thing before I can do the rest of the projection boundary. The projection boundary is in `dev/migrie/interactivity-projection-000`, which is almost done now.

dev/migrie/oop-scratch-4 has the final state of the OOP prototyping on this element. In that branch,

  • HostAndProcess::CreateSwapChain creates a swapchain with DCompositionCreateSurfaceHandle.
  • It attaches it to the SwapChainPanel with ISwapChainPanelNative2::SetSwapChainHandle.
  • It duplicates that handle to the content process, and calls HostClass::ThisIsInsane
  • Later on, the HostClass passes that handle to the DxEngine

So we'll need to do basically that, but a little different, to handle all the actual edge cases we have.

pre-1.10 era work
  • First, update DxEngine to accept the handle, but don't otherwise change anything (Use DComp surface handle for Swap Chain management #10023)
    • Then, change TermControl to create the handle with DCompositionCreateSurfaceHandle, hook it up to the SwapChainPanel with ISwapChainPanelNative2::SetSwapChainHandle.
      • DxEngine::_CreateDeviceResources will need to trigger a callback in the Core, that will then ask the TermControl to create a new swapchain for it.
      • Wait no that's wrong. the DxEngine will always be making the swapchains, because it knows when it needs a new one.
    • DxEngine will always make new swapchains, when needed, with DCompositionCreateSurfaceHandle. It'll raise an event when that happens. The Core will handle that, then raise an event the control will listen to. The Control will handle that event, then ask the Core for the new swapchain HANDLE, as a uint64_t. The Control will use that uint64_t to attach to the SwapChainPanel via ISwapChainPanelNative2::SetSwapChainHandle.
  • Then we'll come back to the rest of the projection
  • At this point, we should make a sample app that's just a grid with two spaces in it. One for an in-proc term control, and one for an out of proc one. We won't do the OOP one yet.
  • Merge Fix a number of shutdown crashes in TermControl #10115 into Add a Scratch.sln for prototyping #10067
  • Enable the Connection able to be created OOP from the rest of the TerminalPage. The content process will be the one to instantiate the connection.
    • The Core is passed a (guid? String?) of a type of connection to spawn, and an IConnectionSettings object. The Core instantiates a type of connection based on that guid/string, and passes the IConnectionSettings from the window process to that new constructor.
    • How does the azure connection work? I think that one's implemented in TerminalApp. If it's a String, we could pass the winrt name, ITerminalConnection conn = RoCreateInstance("TerminalApp.AzureConnection"); conn.Create(iConnectionSettings); , right?
      • No, turns out the Azure connection is in TerminalConnection. So this was actually fine the whole time.
      • Whatever, we're keeping the RoGetActivationFactory thing though, for future needs.
    • How does the debug tap work?
    • Switch from RoActivateInstance&Initialize to RoGetActivationFactory

      • Even neglecting the Windows Runtime, this isn’t even possible in C++ or C#.

    • Reattach event handlers that I disabled for performance's sake
    • Merge Fix a number of shutdown crashes in TermControl #10115 into this branch, or into dev/migrie/oop/connection-factory-rebase, which is attempting to rebase these changes onto dev/migrie/oop/sample-project
    • 📝 in dev/migrie/oop/connection-factory
  • 🦶 PUNTED Move the Core, the Interactivity, into Core.dll. We'll need that for:
    • We may not actually need this. It'd be nice to not have to load all the settings, but we could live with it.

Metadata

Metadata

Assignees

Labels

Area-UserInterfaceIssues pertaining to the user interface of the Console or TerminalIn-PRThis issue has a related PRIssue-ScenarioNeeds-Tag-FixDoesn't match tag requirementsProduct-TerminalThe new Windows Terminal.

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions