Repository navigation
Sill Viewer: Dock badges, bounces and minimised windows in the remote apps' Dock icons; full speed in the background (builds on #66) - #72
Open
ashleigh-hopkins wants to merge 103 commits into
Conversation
…s its own The Mac client (Sources/MacClient, product SillClient, Scripts/make-client-app.sh): the iOS session code compiled unchanged from iOSClient through Shared/ symlinks, with ClientPlatform and HEVCSampleFeed shared by both apps. Each host window opens as a borderless window of its own, streamed at once (side streams, kinds 29–31), with a handle for this Mac's controls, the remote app's menus in the menu bar, a Dock icon per remote app, the clipboard both ways (kind 32), sound (kind 33) and files dropped in and dragged out (kinds 34–37). Input: pointer and all three buttons, trackpad scrolling with momentum, keys, ⌘Tab and ⌘Space when captured, gestures as the host's shortcuts. The host keeps those windows on one shared hidden virtual display (DeskStages), sized to the viewer's largest screen, taps the shown apps' sound and mutes it there (AppAudio), and takes and gives files (FileDrops, DragOut). Every addition goes only to a device that asks or a host that says it takes it; the iOS app is unchanged on the wire. The iPhone build from Xcode 27.0 can leave the 27.1 hinge API out (SILL_NO_DUO_SDK). New checks: mac-keys and main-router; protocol, compatibility, menus and pointer-control know kinds 29–37.
…wer window Two virtual displays with one vendor and product ID are one monitor to macOS, which takes the older offline when the newer comes; that, not a hardware limit, is why windows on displays of their own fell back to the host's screen. Each window's display now has its own identity from a pool (product 0x100 + slot, a serial kept per slot, so macOS remembers how to show each), and the shared display is gone: a window going full screen takes only its own display, never the other streamed windows. While a window is full screen, its display takes the size of the device's window live (VirtualDisplay.resize keeps the display and its ID), so the full-screen Space fills the viewer window and nothing more; leaving full screen gives the display back its size.
A display fitted to a full-screen window is smaller than it was made, and the next placement read that as too small and made it again, which threw the window out of full screen and round again (black, the sound skipping). A display is reused up to the size it was made at, and a window is fitted from wherever it streams: to the device's window as the main stream, to the size it had before full screen beside it. Each window's display is 2× again unless the window itself is larger than a 2× display can be: the device's widest screen made every display 1× before, soft on a Retina screen.
…de once, back to 2× A host file dragged out of one Sill window and dropped on another is dropped on the host from where it is (FileDrop's new `offer` and `window`): a real drag of the real file, so Finder moves it as it would there, and nothing goes to this Mac and back. Every Sill window takes drops: files from this Mac on a window beside the streamed one land in that window. A window's display was made twice: its first size was the device's screen, the placement then asked for that plus room for the menu bar and made it again. It is made once, at the placement's size. A window larger than a 2× display gets a 1× one, and once it is 5 % under the 2× size again it goes back to 2×.
… windows A full-screen window is its display's size, and that size was taken as the window's: the display was made bigger, which threw the window out of full screen and round again. A window full screen (by Accessibility's word, from the first placement) is sized by the size it had before, and fitted to the device's window only by a size its display can take. The viewer opens no app's untitled strips by itself (a browser's full-screen bar). A drag out of a window counts as leaving it when another window is under the pointer, not only outside its rectangle: a drop on a Sill window lying over the first one went back into the first.
A window full screen on its desk display cannot be moved home, and when the display went its Space went to another of Sill's displays, out of sight: a browser looked closed and would not come back after a quit. Leaving the desk now takes it out of full screen first (1.5 s at most), at Quit too.
Only windows on screen were listed, so a window in another Space (a full-screen one, another desktop) could not be reached from a device: a browser stuck full screen on a display out of sight looked gone. Titled windows of regular apps out of sight are listed as well, from the full list the catalog already fetched for its staged windows (the log line for those is gone with it).
…background The streamed desk window going into or out of full screen is seen by a 0.25 s look at it, where the catalog's 2 s poll left the full-screen window at the wrong size for seconds. A full-screen window shown beside the streamed one keeps the size its display was fitted to: it fell back to its size before full screen, a smaller picture, as soon as another window was clicked. The viewer logs each step of a drop on one of its windows.
A right button whose up never reached the host stayed down there: every move was its drag and left clicks read as right clicks until the next right click. A button press now lets go of any other button the host still holds first, and says so in the log.
docs/mac-client-plan.md is the design and justification as the branch stands (the wire, Sill's desk, sound, files, security notes, what was verified, what is open); CLAUDE.md's entry and Layout and DEVELOPMENT.md's section follow it. The protocol check covers every new payload (round-trips and refusals, 209 checks, 5 new mutants, 28 of 28 caught). A drag offer serves only the device it was made to, and a file's parts are taken only from the device that started it; both are dropped when that device leaves (an open file handle and the drag watch's timer outlived it). The viewer's menu loses Virtual Display, and a Mac viewer's windows always go to Sill's desk rather than the iPad's staging. The desk display's functions lose the "shared" names left from the shared-display phase. One compiler warning fixed.
A folder dropped on a Sill window, or dragged out of one, goes as one zip (FolderZip: ditto, the archive Finder makes, with resource forks, extended attributes and symbolic links) and is unpacked where it lands: on the host beside the dropped files, here where the promise is fulfilled. A folder dragged out onto the Trash goes to the host's Trash whole. The protocol check zips and unpacks a folder with a nested file and a link (215 checks).
…Sill windows Between two windows of one size the router tells the new stream apart only by the host's word that it switched, which reached it on the main thread, after the new stream's first sets could: the new window's picture went into the old one and stayed while nothing there redrew (two Finder windows swapping). The window list's source now marks the switch on the network queue, in the stream's own order (MainRouter.listed; main-router 21 checks, 12 of 12 mutants). A drag out allowed no operation inside Sill Viewer, so another Sill window was never offered the drop (no +): it allows copy and move there now, and the other Mac drops its own files.
…o longer takes them) NSImageView registers for file and image drags and, not editable, refuses them; as the deepest view under the pointer it took every drag meant for the window's DropContainer, so another Sill window never offered a drop.
A drop used to reach the other Mac only where it was let go: its app saw no drag, so it could not show where the files would land (a folder, a web page's drop zone, the address bar), and this Mac's modifier keys did nothing. Now, to a host whose list says `liveDrag`: - The host's own files, dragged out of one of its windows: its drag stays under way, the button held. Over any Sill window its pointer follows onto that window (each on its own desk display); let go there, it lands as a drag of its own (Finder moves, ⌥ copies, ⌥⌘ makes an alias). Let go anywhere else (this Mac's Finder, the Trash), the host gives its drag up with Esc. - This Mac's files over a Sill window: sent at once, the host starts its own drag of them under the pointer as they arrive and follows it; dragged away for 0.35 s, given up and the copies removed. Host: LiveDrags (through the coordinator's deliver: the injector's held buttons and keys, the modifier keys pressed as the device holds them), FileDrops' panel and received files split out. Viewer: LiveDrop, the stream view's and DropContainer's drag hooks, the upload split from the drop. A host without it gets kind 35 at the drop, as before. Checks: protocol (LiveDrag round-trips, the list's flag), compatibility, menus and pointer-control (38 taken, 39 unknown; their mutants that renumbered a kind onto 38 use 39).
…n the Dock - The remote app's icon carries Sill Viewer's in its corner, so it never reads as this Mac's app; written again when the icon arrives after the Dock icon started (it used to stay plain). - The Dock menu marks the windows as macOS marks an app's own: ✓ the app's front window, ◆ one minimized; it follows focus and minimizing. - Kept in the Dock and clicked with nothing open: the copy knows its app (SillRemoteApp in its Info.plist), opens Sill Viewer if need be and asks for the app, whose windows on the other Mac open here, or which is launched there when it has none. Docs: the plan and CLAUDE.md name kind 38 and the live drag, its verification and what is open.
Beside the streamed window no fit is asked; the fallback to the size before full screen was taken before the check that keeps a fitted display, so the display shrank and the video filled a quarter of its window, pinned to a corner, whenever another Sill window took focus.
…ts windows, the icon kept - Kind 39 (DockMenu, `dockMenu` in the list): the host reads an app's own Dock menu (Finder's New Finder Window…, a chat app's New Chat) through Accessibility, which has it only while the Dock shows it, so it shows for about half a second there; read without the app's windows and the Dock's own items, and pressed by its place and title. The Dock icon here adds it under its windows, read again at most every 20 s as its menu opens. - Quit from a Dock icon closes that app's windows here (the app runs on there), where it left them open. - The badged icon is also the copy's own (AppIcon.icns, signed again), so one kept in the Dock keeps its icon while not running instead of a blank one. - A full-screen window in the background is captured from its display: an app reports its full-screen window far wider than its Space, and captured alone the video was a quarter of the picture, in a corner. Checks: protocol (DockMenu round-trips), compatibility, menus and pointer-control (39 taken, 40 unknown).
… not a count - A device asks for the sound compressed (SideStreamRequest.audioCodec "aac-eld"): the host encodes the tapped apps' sound as AAC-ELD, about 96 kbit/s at 48 kHz stereo instead of plain PCM's 1.5 Mbit/s, for about 15 ms more delay; a compressed kind 33 carries the decoder's cookie and is never read as plain samples (the channels byte's high bit). Sill Viewer asks for it unless `SillClient.plainAudio` is set; a host that cannot encode, or a device that asks for none, gets PCM. - Sill's desk takes a new window while its displays leave room for one more within a quarter of this Mac's memory (each display holds its own framebuffer, about 130 MB at the largest Retina size); display identities are taken as needed, no longer a pool of 16. Checks: protocol (the compressed packet, the codec asked for).
… the background; full-screen windows reachable after a restart - A button let go where the stream view never hears of it (it moved to another window while the button was held, as the host switched streams) went nowhere, and the other Mac kept it down: every later click read as that button's until it was pressed again (a left click acting as a right click). Every up in the app is looked at first, and one the view would miss is sent from there; a button the view thinks down that this Mac has let go of is let go of at the next move. - A press on a window in the background brings it forward and, still held once the other Mac streams it, goes on there: a drag out of a window in the background starts with one press. - A full-screen window leaves full screen before it goes home and only then is its display removed (an app reports leaving at once, while its window is still the display's size: removed then, the Space outlived the display on the Mac's own, where Accessibility cannot reach it, and after a restart it streamed black and took no input). A window found in such a Space is reached by bringing its app forward, taken out of full screen, and moved to the desk.
…ached only for a pick, Dock menus only of apps - The windows the viewer asked the host to keep live were read from AppKit's list while a window closed, the closing one still in it: the host kept that window on its desk and captured it, the sharing indicator on, and it came back at the next launch. Only windows still open are asked for and kept. - A window in another Space is reached by bringing its app forward only for the window a device picked, at most once a minute (it fired for a side stream's window in another Space and brought its app forward on the host by itself). - An app's Dock menu is read from its own Dock item only (AXApplicationDockItem): a folder or stack of the same name gave the Dock's own menu (Sort by, Display as).
…n helper windows left out; Dock names - A minimized window is taken out of the Dock before it moves to the desk (moved while minimized it never showed there: "the window vanished while being staged"), and minimized again going home. - A picked window in another Space (another desktop) has its app brought forward first, which shows that Space, so it can be moved from it; full screen there, it leaves full screen first. Once a minute a window at most; never for a minimized window or a side stream's. - A window that could not come to the desk and is not on screen is captured on its own, never cropped from its display "with its app": that showed whatever lay there (another app's window) instead. - Windows above every display (an app's hidden helper at y -360) are no longer listed. - A Dock icon is named after its app: the window's app name, else the installed app's, never its bundle ID while either is known.
…t showed the bundle ID, the copy's file name)
…an app's menu; the ◆ it ignored)
…pp with no window gets one - The Desktop source took the first display ScreenCaptureKit listed, which could be one of Sill's hidden desk displays (the Desktop showed a window staged there, not the Mac's screen). It is one of the Mac's own displays now (never VirtualDisplay's vendor), the main one unless a device picked another: the window list names them (`displays`, `desktopDisplay`) and kind 40 (PickDisplay) picks one, the Desktop restarting on it. Sill Viewer lists them under Desktop in its menu bar icon. WindowCatalog's preferMainDisplay, which kept the iPad's Virtual Display off the Desktop, is gone: the rule covers it. - Opening an app that is running with no window (Finder with none open) did nothing: its Dock icon is clicked now, as a person would, which asks it for a window, and the new window streams. Checks: protocol (the displays and PickDisplay), compatibility, menus and pointer-control (40 taken, 41 unknown).
… in a Space - Sill Viewer shows each of the host's own displays as a Desktop window of its own (the menu bar icon lists them). The focused one streams as the Desktop on its display (PickDisplay, then the Desktop); the others stream beside it as side streams, a display's ID with the top bit set where a window ID goes (DesktopWindow), which the host captures whole. Drops and live drags on a Desktop window land on its display. A host that lists no displays keeps the one Desktop window. - A window off screen is listed when it belongs to a Space (another desktop, a full-screen Space, minimized from one), any app's, a menu bar app's too (its minimized window): an app's own hidden windows belong to none (a chat app's hidden helper, which no position test told apart: it moved onto the screen). SkyLight's answer, read-only, looked up at run time.
…haring on the host lets the devices go
- Sill Viewer never sends on input this Mac's own Sill host posted (the event's posting process):
with a viewer on each Mac showing the other's Desktop, a move went to the other Mac, landed on its
viewer, came back as that host's posted move, and neither pointer could be moved.
- A Desktop stream leaves out Sill Viewer's windows as it leaves out Sill's own: they show other
Macs.
- The person at the host stopping the sharing (the menu bar's screen-sharing item) stops every
stream and lets every device go with a goodbye ("stopped", the Mac's words, no reconnect), where
the main stream only went to nothing and the side streams were tried again.
…ng it forward there
- Sill's menu on the host lists the windows shared now ("Shared Windows"), each with Stop Sharing,
and Stop Sharing All: the window goes home, stops streaming, and the devices are told (kind 41,
TakenBack) and close their windows of it; they stay connected and may open it again.
- An app brought forward by the person at the host (its Dock icon, ⌘Tab, a click) whose windows a
device shows has them taken back the same way, unless a device acted in the last 2 s (its own
clicks, picks and menus bring apps forward too); a skipped one is logged with the device's last
message.
- The system's Stop Sharing takes every window back the same way, the devices staying connected,
where it let every device go.
- The Virtual Display item says what it is for: iPhone and iPad; a Mac viewer always moves windows.
Checks: protocol (TakenBack), compatibility, menus and pointer-control (41 taken, 42 unknown).
…both sides ci.yml's mutants list and Tests/checks/README.md both sides' checks (key-strokes 348 and 82 with the middle button's case); select() sets both `selecting` and main's missingPolls reset; the exit stops every capture before restoreFits puts sized windows back and the displays go; StreamClient takes Saffsanity#69's foreground rule through ClientPlatform (a Mac app is never in the background) and the Mac viewer still never asks for the Desktop by itself; CLAUDE.md keeps both Current step entries.
…ts: JSONEncoder's key order differs from one encoding to the next, so the byte comparison failed at random
Owner
|
From a pass over all the open pull requests (nothing is decided yet): this builds on #66 at 9f7153f, four commits behind #66's current head, so it conflicts with #66 in CLAUDE.md and |
This was referenced Oct 15, 2026
grub-basket
added a commit
to grub-basket/sill
that referenced
this pull request
Oct 15, 2026
Review of Saffsanity#111 (the repository owner): Saffsanity#66, and Saffsanity#72 and Saffsanity#73 built on it, add windows from other Spaces and minimized windows to the catalog, so a window can leave the list and come back. The first version of this pull request counted it as new each time: the host dropped a window from its first-seen order when it left and put it first again when it came back, and the device compared each list only with the one before. Every switch of Space or every minimize would have moved windows to the front, on every device, and on Saffsanity#66's Mac viewer too (which compiles StreamClient.swift). - WindowOrder (Sources/StreamProtocol, pure, shared by both ends): seen() is the host's first-seen order: windows never listed before first, in the order listed, then every window seen before, gone ones included, so one that comes back takes its old place; past a cap the windows gone the longest are forgotten, never one listed now. arranged() is a device's arrangement after a list: windows this session has never seen, and the arrangement does not hold, go first; nothing on an unarranged bar or on the session's first list. - WindowCatalog keeps every window it has listed (seenCap 1024) and orders by WindowOrder.seen. - StreamClient keeps every window id this session's lists named (seenWindowIDs; cleared at tearDown and when the Mac changes) and takes WindowOrder.arranged's result, saving it. Tests/checks/window-order: 14 checks, 10 of 10 mutants caught; README and CI's mutants matrix. Tested: swift build -c release; the check and its mutants; the iOS app type-checks in a scratch copy with DuoPostureReader's iOS 27.1 calls stubbed (this Mac has the iOS 27.0 SDK only). Not run on a device. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the clipboard's kind 32 negotiation and the display teardown work Conflicts kept both sides: the pill's comment has the bolt and the refresh that restarts captures; the protocol mutants list both sides' cases; CLAUDE.md's wire list has the Dock bounces and full-speed flags beside the clipboard's subscription; the plan keeps this branch's notifications line and mac-client's merge notes.
Author
This was referenced Oct 19, 2026
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #66 (Sill Viewer). Until #66 merges, GitHub shows its commits here too; this PR's own are the ones after it:
Why
Whether Sill Viewer could forward the host's notifications and Dock bounces, measured before building the forwarding:
docs/notifications-spike.md. Every probe was a throwaway and has been removed. Five pieces are built on it.Built
LSWantsAttention, every 250 ms while a Mac viewer is connected; no permission) and sends them asWindowList.attention, a new optional field an older viewer ignores. The app's Dock icon on the viewing Mac bounces while the host's does. Seen between two Macs: a test app's one-time and repeating bounces both reached the viewer's Dock icon.SideStreamRequest.fullRate, sent only to a host whose window list saysfullRate. The pill's lights and bolt are now Accessibility elements. Seen between two Macs: a marked window in the background at 120 fps and 30 Mbps, unmarked at 15 fps and 3 Mbps.SillDockProxy, an informational request). The first state, and a badge that falls or goes, do not bounce. Seen with the test app: one 1 s bounce for each of three rises.Findings (macOS 27.0.1; the Dock bounce also on 27.0)
LSWantsAttention1 while it asks for attention: briefly for a one-time bounce, until cancelled or activated for a repeating one. Two private, exported calls read it, with no permission at all. The first pass missed this: it took request ID 0 for a failure, and looked only at Accessibility and notifications.Recommendation
Not yet tried: a third full-speed mark replacing the oldest; a real app's bounce (the test apps go through the same AppKit call), plus the notification cases the doc lists.