[Fiber] Don't wait on Suspensey Images if we guess that we don't load them all in time anyway (#34481)

Stacked on #34478.

In general we don't like to deal with timeouts in suspense world. We've
had that in the past but in general it doesn't work well because if you
have a timeout and then give up you made everything wait longer for no
benefit at the end. That's why the recommendation is to remove a
Suspense boundary if you expect it to be fast and add one if you expect
it to be slow. You have to estimate as the developer.

Suspensey images suffer from this same problem. We want to apply
suspensey images to as much as possible so that it's the default to
avoid flashing because if just a few images flash it's still almost as
bad as all of them. However, we do know that it's also very common to
use images and on a slow connection or many images, it's not worth it so
we have the timeout to eventually give up.

However, this means that in cases that are always slow or connections
that are always slow, you're always punished for no reason.

Suspensey images is mainly a polish feature to make high end experiences
on high end connections better but we don't want to unnecessarily punish
all slow connections in the process or things like lots of images below
the viewport.

This PR adds an estimate for whether or not we'll likely be able to load
all the images within the timeout on a high end enough connection. If
not, we'll still do a short suspend (unless we've already exceeded the
wait time adjusted for #34478) to allow loading from cache if available.

This estimate is based on two heuristics:

1) We compute an estimated bandwidth available on the current device in
mbps. This is computed from performance entries that have loaded static
resources already on the site. E.g. this can be other images, css, or
scripts. We see how long they took. If we don't have any entries (or if
they're all cross-origin in Safari) we fallback to
`navigator.connection.downlink` in Chrome or a 5mbps default in
Firefox/Safari.
2) To estimate how many bytes we'll have to download we use the
width/height props of the img tag if available (or a 100 pixel default)
times the device pixel ratio. We assume that a good img implementation
downloads proper resolution image for the device and defines a
width/height up front to avoid layout trash. Then we estimate that it
takes about 0.25 bytes per pixel which is somewhat conservative
estimate.

This is somewhat conservative given that the image could've been
preloaded and be better compressed.

So it really only kicks in for high end connections that are known to
load fast.

In a follow up, we can add an additional wait for View Transitions that
does the same estimate but only for the images that turn out to be in
viewport.

Author: sebmarkbage

fixtures/view-transition/src/components/Page.js

@@ -50,7 +50,8 @@ function Component() {
       <p>
         <img
           src="https://react.dev/_next/image?url=%2Fimages%2Fteam%2Fsebmarkbage.jpg&w=3840&q=75"
-          width="300"
+          width="400"
+          height="248"
         />
       </p>
     </ViewTransition>

packages/react-dom-bindings/src/client/ReactFiberConfigDOM.js

@@ -143,6 +143,7 @@ import ReactDOMSharedInternals from 'shared/ReactDOMSharedInternals';
 export {default as rendererVersion} from 'shared/ReactVersion';
 
 import noop from 'shared/noop';
+import estimateBandwidth from './estimateBandwidth';
 
 export const rendererPackageName = 'react-dom';
 export const extraDevToolsConfig = null;
@@ -5907,6 +5908,7 @@ type SuspendedState = {
   stylesheets: null | Map<StylesheetResource, HoistableRoot>,
   count: number, // suspensey css and active view transitions
   imgCount: number, // suspensey images
+  imgBytes: number, // number of bytes we estimate needing to download
   waitingForImages: boolean, // false when we're no longer blocking on images
   unsuspend: null | (() => void),
 };
@@ -5917,6 +5919,7 @@ export function startSuspendingCommit(): void {
     stylesheets: null,
     count: 0,
     imgCount: 0,
+    imgBytes: 0,
     waitingForImages: true,
     // We use a noop function when we begin suspending because if possible we want the
     // waitfor step to finish synchronously. If it doesn't we'll return a function to
@@ -5926,10 +5929,6 @@ export function startSuspendingCommit(): void {
   };
 }
 
-const SUSPENSEY_STYLESHEET_TIMEOUT = 60000;
-
-const SUSPENSEY_IMAGE_TIMEOUT = 500;
-
 export function suspendInstance(
   instance: Instance,
   type: Type,
@@ -5953,6 +5952,18 @@ export function suspendInstance(
     // The loading should have already started at this point, so it should be enough to
     // just call decode() which should also wait for the data to finish loading.
     state.imgCount++;
+    // Estimate the byte size that we're about to download based on the width/height
+    // specified in the props. This is best practice to know ahead of time but if it's
+    // unspecified we'll fallback to a guess of 100x100 pixels.
+    if (!(instance: any).complete) {
+      const width: number = (instance: any).width || 100;
+      const height: number = (instance: any).height || 100;
+      const pixelRatio: number =
+        typeof devicePixelRatio === 'number' ? devicePixelRatio : 1;
+      const pixelsToDownload = width * height * pixelRatio;
+      const AVERAGE_BYTE_PER_PIXEL = 0.25;
+      state.imgBytes += pixelsToDownload * AVERAGE_BYTE_PER_PIXEL;
+    }
     const ping = onUnsuspendImg.bind(state);
     // $FlowFixMe[prop-missing]
     instance.decode().then(ping, ping);
@@ -6070,6 +6081,14 @@ export function suspendOnActiveViewTransition(rootContainer: Container): void {
   activeViewTransition.finished.then(ping, ping);
 }
 
+const SUSPENSEY_STYLESHEET_TIMEOUT = 60000;
+
+const SUSPENSEY_IMAGE_TIMEOUT = 800;
+
+const SUSPENSEY_IMAGE_TIME_ESTIMATE = 500;
+
+let estimatedBytesWithinLimit: number = 0;
+
 export function waitForCommitToBeReady(
   timeoutOffset: number,
 ): null | ((() => void) => () => void) {
@@ -6109,6 +6128,18 @@ export function waitForCommitToBeReady(
         }
       }, SUSPENSEY_STYLESHEET_TIMEOUT + timeoutOffset);
 
+      if (state.imgBytes > 0 && estimatedBytesWithinLimit === 0) {
+        // Estimate how many bytes we can download in 500ms.
+        const mbps = estimateBandwidth();
+        estimatedBytesWithinLimit = mbps * 125 * SUSPENSEY_IMAGE_TIME_ESTIMATE;
+      }
+      // If we have more images to download than we expect to fit in the timeout, then
+      // don't wait for images longer than 50ms. The 50ms lets us still do decoding and
+      // hitting caches if it turns out that they're already in the HTTP cache.
+      const imgTimeout =
+        state.imgBytes > estimatedBytesWithinLimit
+          ? 50
+          : SUSPENSEY_IMAGE_TIMEOUT;
       const imgTimer = setTimeout(() => {
         // We're no longer blocked on images. If CSS resolves after this we can commit.
         state.waitingForImages = false;
@@ -6122,7 +6153,7 @@ export function waitForCommitToBeReady(
             unsuspend();
           }
         }
-      }, SUSPENSEY_IMAGE_TIMEOUT + timeoutOffset);
+      }, imgTimeout + timeoutOffset);
 
       state.unsuspend = commit;
 

packages/react-dom-bindings/src/client/estimateBandwidth.js

@@ -0,0 +1,112 @@
+/**
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ *
+ * @flow
+ */
+
+function isLikelyStaticResource(initiatorType: string) {
+  switch (initiatorType) {
+    case 'css':
+    case 'script':
+    case 'font':
+    case 'img':
+    case 'image':
+    case 'input':
+    case 'link':
+      return true;
+    default:
+      return false;
+  }
+}
+
+export default function estimateBandwidth(): number {
+  // Estimate the current bandwidth for downloading static resources given resources already
+  // loaded.
+  // $FlowFixMe[method-unbinding]
+  if (typeof performance.getEntriesByType === 'function') {
+    let count = 0;
+    let bits = 0;
+    const resourceEntries = performance.getEntriesByType('resource');
+    for (let i = 0; i < resourceEntries.length; i++) {
+      const entry = resourceEntries[i];
+      // $FlowFixMe[prop-missing]
+      const transferSize: number = entry.transferSize;
+      // $FlowFixMe[prop-missing]
+      const initiatorType: string = entry.initiatorType;
+      const duration = entry.duration;
+      if (
+        !transferSize ||
+        !duration ||
+        !isLikelyStaticResource(initiatorType)
+      ) {
+        // Skip cached, cross-orgin entries and resources likely to be dynamically generated.
+        continue;
+      }
+      // Find any overlapping entries that were transferring at the same time since the total
+      // bps at the time will include those bytes.
+      let overlappingBytes = 0;
+      // $FlowFixMe[prop-missing]
+      const parentEndTime: number = entry.responseEnd;
+      let j;
+      for (j = i + 1; j < resourceEntries.length; j++) {
+        const overlapEntry = resourceEntries[j];
+        const overlapStartTime = overlapEntry.startTime;
+        if (overlapStartTime > parentEndTime) {
+          break;
+        }
+        // $FlowFixMe[prop-missing]
+        const overlapTransferSize: number = overlapEntry.transferSize;
+        // $FlowFixMe[prop-missing]
+        const overlapInitiatorType: string = overlapEntry.initiatorType;
+        if (
+          !overlapTransferSize ||
+          !isLikelyStaticResource(overlapInitiatorType)
+        ) {
+          // Skip cached, cross-orgin entries and resources likely to be dynamically generated.
+          continue;
+        }
+        // $FlowFixMe[prop-missing]
+        const overlapEndTime: number = overlapEntry.responseEnd;
+        const overlapFactor =
+          overlapEndTime < parentEndTime
+            ? 1
+            : (parentEndTime - overlapStartTime) /
+              (overlapEndTime - overlapStartTime);
+        overlappingBytes += overlapTransferSize * overlapFactor;
+      }
+      // Skip past any entries we already considered overlapping. Otherwise we'd have to go
+      // back to consider previous entries when we then handled them.
+      i = j - 1;
+
+      const bps =
+        ((transferSize + overlappingBytes) * 8) / (entry.duration / 1000);
+      bits += bps;
+      count++;
+      if (count > 10) {
+        // We have enough to get an average.
+        break;
+      }
+    }
+    if (count > 0) {
+      return bits / count / 1e6;
+    }
+  }
+
+  // Fallback to the navigator.connection estimate if available
+  // $FlowFixMe[prop-missing]
+  if (navigator.connection) {
+    // $FlowFixMe
+    const downlink: ?number = navigator.connection.downlink;
+    if (typeof downlink === 'number') {
+      return downlink;
+    }
+  }
+
+  // Otherwise, use a default of 5mbps to compute heuristics.
+  // This can happen commonly in Safari if all static resources and images are loaded
+  // cross-orgin.
+  return 5;
+}