[Master feature] Element-level infinite scroll

[ericlindley-g]

Feature description:

Support element-level infinite scroll, such that when the user reaches the end of a list of items (search results, product cards, etc), a call can be made to an arbitrary endpoint to populate the list with more items. For the best user experience, users should never see a loading spinner, but instead should either be presented with already-loaded results, or a button to tap to load more.

Update

This feature is available for origin trial per #20592 as of release #20587.

Related demos

Basic demos
https://amp-infinite-scroll.glitch.me/
https://amp-infinite-scroll.glitch.me/flex.html
Pretty Demo
TODO: Add animation (slide up / slide in / fade in)
TODO: Variable heights
Example with a finite number of pages
https://amp-infinite-scroll.glitch.me/finite.html
Example with bad URL
https://amp-infinite-scroll.glitch.me/bad-url.html
Example with slow loading URL
https://amp-infinite-scroll.glitch.me/latency.html
Use Slow 3G network tab option or 2g poor on any of the examples
Manual load more
Example with finite number of pages
https://amp-infinite-scroll.glitch.me/finite-2.html
Example with bad URL
https://amp-infinite-scroll.glitch.me/bad-url-2.html
Example with slow loading URL
https://amp-infinite-scroll.glitch.me/latency-2.html

Related issues

#19502

Related PRs

#19519
#19399

[ampprojectbot]

This issue doesn't have a category which makes it harder for us to keep track of it. @ericlindley-g Please add an appropriate category.

[mofidsky]

we need this feature urgently. when do you think you can release a version?

[ericlindley-g]

@mofidsky We're targeting release this quarter— @cathyxz, who is working on the implementation, may or may not be able to offer a better estimate.

We'd love to hear feedback when you're able to try it out—if you can share an examples of how you plan to use it that could help us make sure we're capturing relevant use cases.

[mofidsky]

@ericlindley-g We're trying to build a news web site so that contents are loaded when users scroll down, something like Facebook web site or Instagram app for instance.

[ericlindley-g]

@mofidsky — Thanks! That sounds like the general use-case we're building for.

I should note that amp-next-page supports infinite scroll for entire documents of content (as opposed to infinite scroll for elements within a single page), in case you're interested in that experience.

[heiwen]

@ericlindley-g, @cathyxz We are also have a use case for this component on the product detail pages of our e-commerce aggregator. We compare offers from different merchants selling the same product, in the case of marketplaces they even have multiple offers of the same product. We allow the user to browse for the merchant/marketplace first and then drill-down into the respective offer. Right now we are pre-loading all the content into the HTML and just playing with the visibility, that bloats the HTML quite significantly though.

For an example, check out the following page: https://iprice.my/compare/apple-iphone-8/ and scroll to the "Compare Prices" section.

This is quite high in our backlog right now, happy to help in the development or testing of the component, let us know how we can contribute.

[buihoanghai]

My current approach is using the recursive template mustache. This template has 2 parts. First one for loop items, another one for next items. The next section consist a form which using this template again. Check the image bellow:

Im thinking if we add high trust event for amp-position-observer we will have real infinity scroll :).

[aghassemi]

@buihoanghai We are working a first-class support for this. It is under development but you can give it a try experimentally ( not valid in Prod ) using amp-load-more experiment (sample: https://github.com/ampproject/amphtml/blob/master/test/manual/amp-list-load-more.amp.html )
ETA for launch is Dec/Jan.

We are not planning to make position-observer emit hight-trust events as scroll is not a strong user-interaction signal to be allowed to relayout the page.

[ericlindley-g]

Thanks @heiwen — I defer to @cathyxz if additional help would speed up the timeline here, but we appreciate the info on your use case either way

By the way — great AMP-first implementation of your site! It's awesome to see AMP used to build great experiences on the main version of a site like yours

[leonahliang90]

@ericlindley-g Nice to see this feature is on the radar, is there anyway we can contribute and build this thing together to achieve infinite scroll to load more data like this along with Masonry / Waterfall layout like what pinterest is having ? we are looking forward to contribute and share the workload together, but with concern of not knowing what's the current state for these components and how to step in. Please advise.

/cc @cathyxz

[ericlindley-g]

Thanks for reaching out, @leonalicious — @cathyxz is in the best position to know if there are opportunities to contribute on this component, so I'll let her answer that aspect. My guess is that the most important place to contribute is in testing the component when it's feature complete and experimentally available, but @cathyxz may know of opportunities to contribute to the development itself.

[cathyxz]

It's experimentally available under amp-list-load-more, with an in-progress demo here. I'm trying to resolve the amp-list-resizable-children right now, and will be iterating on various corner cases for infinite scroll after that.

[leonahliang90]

It's experimentally available under amp-list-load-more, with an in-progress demo here. I'm trying to resolve the amp-list-resizable-children right now, and will be iterating on various corner cases for infinite scroll after that.

Cool, any timeline for the infinite scroll ? May I assist you ? which branch are you using for development of this feature ?

[leonahliang90]

@cathyxz good day there ! it has been a while since we last heard the update from you, any progress / assistance I may offer please?

[trjones1]

@cathyxz we noticed an issue perhaps with iPhone and infinite scrolling.

In development load more works fine for me (windows/chrome) however when I make a push to our QA servers, we never get to load any of the additional pages.

I have been able to replicate this on the example that you've provided.
https://amp-infinite-scroll.glitch.me/

This was tested on an iPhone X, iOS version 12.1.4

Please advise, as we are planning on taking this live very soon. I was previously using the experimental version of this feature using <div load-more-loading> until today when I implemented the official <amp-list-load-more> syntax.

The experimental version worked, but does not pass the AMP validation (and therefore caching) due to added JS for opting in.

PS - On my IT Director's Samsung Galaxy S9+, our paging worked just fine. Also on my coworker's Samsung Galaxy S9+, it did not work, neither did the demo you've provided.

iPhone Test:

Coworker's S9+ test.

[cathyxz]

Hi @trjones1, thanks for reporting. I've taken a look, and the demo operates correctly in a browser (tested on Safari iOS X) when I have the amp-list-load-more experiment enabled via the experiments page. Because the feature is experimental, users have to opt their browsers into the experiment in order to turn it on. My demo page is also https-only (I get the same error if I navigate to it in http). Without an origin trial token, pages using experimental features aren't valid AMP. Since the demo on glitch is showing unable to load more, which is the load-failed-element, this likely means that the load-more url returned a 404. I suspect it is because the page was loaded in http. If not, could you send a screenshot of your js console or network tab when this happens?

If this happens on your deployed website, I would double check that your cors setup is correct. See https://www.ampproject.org/docs/fundamentals/amp-cors-requests for specifics.

[trjones1]

Thanks, @cathyxz - you're right. I visited the site via http on my phone and that reproduced the error.

I'm unable to make this work on our website when not using the experimental features. I am using the following markup:
<amp-list-load-more load-more-loading>Loading more </amp-list-load-more>
<amp-list-load-more load-more-end>No more products to show...</amp-list-load-more>
<amp-list-load-more load-more-button class="hide">Click here to see more...</amp-list-load-more>
<amp-list-load-more load-more-failed>Failed to load more...</amp-list-load-more>

However, with experimental features turned on, and using the following markup, it works:
<div load-more-loading>Loading more styles...</div>
<div load-more-button class="hide">Click here to see more...</div>
<div load-more-failed>Something went wrong...</div>

The unfortunate news is, the latter option makes us fail AMP validation - due to having javascript load when we opt in for the experimental features:

<meta name="amp-experiments-opt-in" content="amp-list-load-more">
<script>
(self.AMP=self.AMP||[]).push(function(AMP){
AMP.toggleExperiment('amp-list-load-more', true);
AMP.toggleExperiment('amp-list-resizable-children', true);
});
</script>

Non-Experimental works on my development machine too! (visiting the deployed website) So I cannot really give you a screenshot of my network tab since everything works. It happens strictly on device for me.

I can PM you the dev site link if you'd like to observe for yourself.

[trjones1]

@cathyxz You get the same issue I do when it comes to passing validation:

Does this prevent the page from being entered into the Google AMP Cache? Will we not have a thunderbolt as a result?

Thanks again.

[cathyxz]

Yup @trjones1. You need an origin trial token (specific to your domain) to be able to launch this in valid AMP. Can you ping me on slack (czhu)?

[cathyxz]

So for anyone else who'd like to give this a try, the origin trial enrolment form is here.
/cc @nainar

[leonahliang90]

@cathyxz Good News Cathy, we have finally gone online with Infinite scroll last week, and have steadily increased the traffic to 50% up to today date, seems everything is so cool! thank you for the effort and hard work, cheers !

[cathyxz]

Thank you @leonalicious for working with us through this 47-comment-long thread and helping us iron out all the bugs!

[cathyxz]

Just a heads up to everyone waiting for an origin trial token, we're launching this with #21355. If you want to use it, just keep track of the release issues to see when #21355 makes it to production.

[clodisewp]

@cathyxz Sorry for bothering you again. This time, I want to ask if I can got the amp-list response outside the tag. I tried to get it by id, but it does not work. I just do not want to request for the second time in the same page.

[cathyxz]

Hi @clodisewp, sorry but your question is a bit ambiguous, and I'd rather not waste cycles making poor guesses at what you're trying to ask. Just in case this is related to using amp-state as a src for amp-list and the fact that currently two XHRs are being made, there is currently an open FR and design discussion here #20752.

If your question here is also not a bug or a feature request, is there any chance you could ask your question on stackoverflow and link to it in our slack channel? I think that's a much better avenue for getting questions answered on how to use AMP components (others with the same question will also benefit), and people will likely respond to you faster there too. =)

[clodisewp]

Hi @clodisewp, sorry but your question is a bit ambiguous, and I'd rather not waste cycles making poor guesses at what you're trying to ask. Just in case this is related to using amp-state as a src for amp-list and the fact that currently two XHRs are being made, there is currently an open FR and design discussion here #20752.

If your question here is also not a bug or a feature request, is there any chance you could ask your question on stackoverflow and link to it in our slack channel? I think that's a much better avenue for getting questions answered on how to use AMP components (others with the same question will also benefit), and people will likely respond to you faster there too. =)

@cathyxz Sorry for my poor English.
The open FR #20752 is not what I need, and stackoverflow is an oversea website for me.
Now I describe my question with text and imge:

The following is the json in one request:

This is the data distribution:

You can see the place B is outside of amp-list. Is there any solution for this problem under the conditon of only one request?
Thanks!

[xiaotong-kk9]

Hi @cathyxz
With dev-channel version 1903191703470, visit this page.
Scroll down to Personal Flow Part(below 15 Baby & Kids), click view-more.
The height of amp-list is wrong just like below.

Because this issue cannot be reproduced with prd version 1903141753530.
So I suppose this issue is not caused by wrong implementation?

[cathyxz]

Hi @xiaotong-kk9,

Based on a quick look, I think this is caused by hitting our overflow condition. So note that if the bottom of your list is at the bottom of the viewport when you click the load-more button (i.e. move your view more button close to the bottom of your viewport), it will resize correctly. However, if there is significant space in the viewport below your amp-list, it will refuse to resize and push the content below it out of the viewport. I think you are running into the overflow condition discussed in #14060 (comment) (workarounds are in this thread and above).

I haven't diff-ed this against 1903141753530 yet but I think this behavior is expected given current constraints.

[cathyxz]

#21355 is now in production.

[Pkmmte]

@cathyxz Since amp-next-page is still experimental, my company decided to experiment with element-level infinite scroll to incorporate infinite scrolling for articles. I think we got it working on staging, but we're wondering if this violates any AMP rules. Just wanna make sure our pages still index as AMP on search.

Can you please verify that using amp-list for entire AMP articles won't cause us any issues approval-wise?

[cathyxz]

So this feature is now launched, hence any pages using it will be valid AMP, and will index in Search as AMP. However, note that this feature wasn't designed with loading more articles in mind, so its definitely possible that you might run into issues implementation-wise (I currently can't think of a great way to make the code for your use case look very clean...). On the whether this is a good idea question or goes against any AMP best practices, I defer to @nainar.

[nainar]

Hi @Pkmmte what are the use cases you are trying to solve for document level infinite scroll?
For e.g. what analytics support do you need? Are you able to change the URL as the user scrolls into the next article?

[Pkmmte]

@nainar We run an editorial website, and I'm trying to surface a new articles after a user has finished reading the original article without them having to click on any suggestions.

We weren't able to change the URL as the user scrolls, but I ended up using an amp-iframe for the article body anyway, given HTML limitations with amp-mustache. The amp-iframe source is an exact copy of the AMP page, but with a special parameter (?onlyBody=true) to let the server know to only render the article body, but still log it as an analytics page view.

Everything is working as intended on our staging server right now for the most part. It was a bit hacky, but it accomplishes our primary goal. It passed the AMP Validator test, too. We just want to know if we'll get punished by Google for using amp-list in a way that it was not intended for, such as this.

I can provide more details in a private email, if necessary.

[Pkmmte]

@nainar Any further clarification on this?

[Couponkozmy]

Get Coupons, Promo Codes, Coupon Codes and Vouchers for Malaysian Online Stores to save money. [https://www.couponkoz.my/]