Scroll restoration from history state

[daun]

Most routing libraries currently store the scroll position in the history state and manually restore it from there.

We decided to stick with browser auto-restoration for the core, but to have this functionality in the scroll plugin.

Work in progress

First draft, not working properly, doesn't even build 🤠

Description

• Remember scroll position in history state
• Restore scroll position from history state
• Create two hooks: scroll:store when saving position, scroll:restore when restoring position
• Programmatically read and manipulate the stored position
• Control over the exact point in time when to restore the scroll position
• Identical behavior between animated and non-animated (history) visits
• Currently uses debounced scroll function to update position when scrolling stops
  • There is the new scrollend event but it's not 100% supported unfortunately and the polyfill is rather heavy

Prior art

• Turbo: save to history
• pjax: save to history
• Astro: save to history
• Livewire: save to body attribute

To-do

• Restore scroll position on reload
• Check if this breaks anything
• Check if this needs preparation from the core

Checks

• The PR is submitted to the next branch
• The code was linted before pushing (npm run lint)
• All tests are passing (npm run test)
• New or updated tests are included
• The documentation was updated as required

[hirasso]

What use case is the scroll listener for? Can we get away with

• a visit:start handler
• a beforeunload handler for hard reloads

for storing the scroll position(s)?

[daun]

Good question. The main reason I integrated it here is that after some research almost all similar routing libraries do it this way. I remember beforeunload being rather unreliable, but let me do some more research. If we don't need a scroll listener, we should definitely get rid of it.

[hirasso]

I had good success with beforeunload for storing the scroll positions of overflowing divs in other projects.

I'd also say that for something like restoring scroll positions, it's enough if it works 90% of the time :)

[daun]

@hirasso I remember now.

We need the scroll handler for history visits. In the popstate DOM event, the history state is already replaced by a new one, so at that point we can no longer append the scroll position to it. The visit:start hook fires after popstate (or it's triggered by it), so it's already too late in there too. There's no beforeunload for popstate :(

Technically, it would be possible with the new Navigation API, but it's basically Chrome only.

[hirasso]

Couldn't we just do it on visit:start, but appending the scroll positions not to the current (new) URL but rather to the visit.from.url entry? Basically, just like it's done currently with the plain JS object for storing the scroll positions:

scroll-plugin/src/index.ts

Line 276 in 034503e

this.cacheScrollPositions(visit.from.url);

...and then, inside cacheScrollPositions:

// current:
// this.cachedScrollPositions[cacheKey] = positions;
// new: url is coming from `visit.from.url` so it's always the previous one, even during popstate events:
updateHistoryRecord(url, { scrollPositions: positions });

To recap: the window scroll position will have already been updated after popstate, but only if window.history.scrollRestoration === 'auto', in which case we leave the window scroll position alone during popstate, anyways:

// Bail early on popstate if not animated: browser will handle it
if (visit.history.popstate && !visit.animation.animate) {
	return;
}

Two more thoughts:

• The current implementation on main doesn't store the element itself in the scroll positions array, as this is not really needed. A page will always have the same amount of [data-swup-scroll-container] elements, so a simple array is enough.
• I'm still unsure how "preserving" the scroll positions should work with history state entries if shouldResetScrollPosition() returns false. Currently, this is straight forward as scroll positions are stored only once for each URL:

scroll-plugin/src/index.ts

Lines 371 to 374 in 034503e

	resetScrollPositions(url: string): void {
	const cacheKey = this.swup.resolveUrl(url);
	delete this.cachedScrollPositions[cacheKey];
	}

But with history.state, there can be an arbitrary amount of entries for the same URL. This, for example, will create three history entries for the same URL:

history.pushState({page: 1}, "", "/example");
history.pushState({page: 2}, "", "/example");
history.pushState({page: 3}, "", "/example");

History keeps confusing me 🐚😄

If it turns out that preserving the scroll position on forward (link-click) navigation doesn't work with history.state, I'd prefer not to use it and keep the current plain-object solution. Or are there any other benefits that you can think of?

We could still use it for restoring the scroll positions of [data-swup-scroll-container] elements on hard reloads. I've done this in a recent project and it's working beautifully.

All these are only considerations, I'm happy to be proven wrong or right via an actual implementation :)

[hirasso]

Btw, feel free to ignore me until you find time :)

[daun]

This is all theoretical, so some of this might not make much sense 🤠 Haven't touched the branch in a while.

Couldn't we just do it on visit:start, but appending the scroll positions not to the current (new) URL but rather to the visit.from.url entry? Basically, just like it's done currently with the plain JS object for storing the scroll positions:

The main goal is to store the scroll positions in history so a) different visits to a URL can have different scroll positions and b) the positions survive a reload. If we put them into the visit object, I don't see much of an improvement there. The visit object is gone after the visit, right?

A page will always have the same amount of [data-swup-scroll-container] elements, so a simple array is enough.

That's also a (minor) goal of this PR. Remembering arbitrary scroll container setups. That would require giving them IDs in those advanced use cases, but e.g. going from the homepage with one scroll container to a filter page with three containers would then be possible technically.

I'm still unsure how "preserving" the scroll positions should work with history state entries if shouldResetScrollPosition() returns false. Currently, this is straight forward as scroll positions are stored only once for each URL:

Good point! Haven't thought about that part too much. I assume it could work by actually preventing the reset while it's happening (or before), instead of removing the data or setting it to 0? But this is very theoretical at this point, hehe.

If it turns out that preserving the scroll position on forward (link-click) navigation doesn't work with history.state, I'd prefer not to use it and keep the current plain-object solution. Or are there any other benefits that you can think of?

A lot of libraries, including Astro, seem to do it this way, so I'm hopeful 🤠

[hirasso]

The main goal is to store the scroll positions in history so a) different visits to a URL can have different scroll positions and b) the positions survive a reload. If we put them into the visit object, I don't see much of an improvement there. The visit object is gone after the visit, right?

That was a misunderstanding. I didn't mean to store the positions in the visit object. Rather to use the url of visit.from.url. But right, if the goal is to support different scroll positions for different visits to the same URL, that's a good point.

[hirasso]

That's also a (minor) goal of this PR. Remembering arbitrary scroll container setups. That would require giving them IDs in those advanced use cases, but e.g. going from the homepage with one scroll container to a filter page with three containers would then be possible technically.

I'd like to discuss this in a call sometime. In my mind, it's not an issue, but it might still well be one 😇

[daun]

Call sounds fantastic 📞

[hirasso]

Notes from our call:

• a debounced scroll handler is indeed the only viable way to store scroll positions reliably, as history.replaceState() can only alter the current history entry and there is no way to alter the previous entry on popstate
• we do not need ids for restoring scroll container positions, as one page will always have the same amount of them. A simple array is enough
• history.scrollRestoration will probably always need to be set to "manual"

Another note from my current project:

Restoring scroll positions after a reload needs to be done after one await tickTick() to make it more reliable.

[hirasso]

As discussed, I (re-)started work on this in another branch (not yet published). The problem I soon stumbled upon:

shouldResetScrollPosition doesn't seem like it's compatible with storing scroll positions in the history stack, because every link creates a new history entry and there's no way to retrieve the information from previous history entries.

So this is the big decision we need to make: **Is the benefit of having different stored scroll positions for each history entry (vs. only for each unique URL) worth dropping support for shouldResetScrollPosition? **

Pro

• Switching to the native history stack would be closer to native behavior
• other pros??

Con

• I'm using that option in a few projects of mine and it will be hard to re-implement it in userland (probably the only way would be via history.go(-n))
• other cons??

Scroll restoration after a page reload could still be implemented via history state on beforeunload, even if we decide to stay close to our current solution with a URL-based cache object.

Happy to discuss.

[hirasso]

maybe there is still a way to keep shouldResetScrollPosition and still switch to history.state:

We could maintain our scroll positions cache just as we have it now, but "mirror" it in history.state. The scroll positions cache would also need to store the URL it is attached to. Then, if shouldResetScrollPosition would evaluate to false, we could pick the first entry in our own stack that matched the target URL. And otherwise use the history state.

No, bad idea.

[daun]

Can you catch me up on what shouldResetScrollPosition does? What's a typical scenario you'd use it for?

[hirasso]

Can you catch me up on what shouldResetScrollPosition does? What's a typical scenario you'd use it for?

https://swup.js.org/plugins/scroll-plugin/#shouldresetscrollposition

You can customize when to reset vs. restore while clicking a link using the shouldResetScrollPosition option. A common use case would be a custom back button: clicking it would normally reset the scoll position to the top while users would expect it to restore the previous scroll position on the page the link points towards.

[hirasso]

Observation: calling history.replaceState on scroll makes my browser stutter, even when debounced.

EDIT: Only Arc, only in a deprecated version of blink.

[hirasso]

Note to future selves: We decided that managing scroll positions via history state is not worth the trouble and would even break features like shouldResetScrollPosition. If users want to do it via history.state, they can use a stand-alone package like @hirasso/restore-scroll:

import Swup from "swup";
import { restoreScroll } from "@hirasso/restore-scroll";

function initPage() {
  document.querySelectorAll(".overflow-y-auto,.overflow-x-auto,.overflow-auto").forEach((el) => restoreScroll(el));
}

initPage();

new Swup({
  hooks: {
    "page:view": initPage,
  },
});