Infinite Scroll WebSites via AutoPagerize - Hacky, but the beginning of something cool
One of the things I like the most about Bing Image search (one of the things I prefer about it) is the "infinite scroll" feature. If you search for an image and start scrolling, it'll just keep going and going, moving the scroll bar each time and appending new images on the bottom. This concept of "infinite scroll" has been called just that, as well as endless pages, autopagerize, etc. There's even a jQuery plugin called Infinite Scroll if you want to enable something like this on your site programmatically.
However, there's also been a quiet revolution on sites, and to some extent, in browsers to make infinite scroll a standard thing. At least, a de facto standard, and you can enable it on your site with minimal effort.
The general idea is that the browser notices that you're scrolling to the end and rather than making you click, it'll fetch the next page via AJAX and append it to the page you're already on. This screenshot from the AutoPagerize for Chrome extension shows it best:
There's a few things needed and it requires a bit of dancing on your part to make it happen.
Enabling Autopagerize as a Browser of the Web
For the longest time Autopagerize has been a "Greasemonkey script." Greasemonkey is an add-on itself that enable others add-ons, via easy scripts, to dramatically change the behavior of your browser. I'm not a huge fan myself, as I have some security concerns. The main site that promotes this is a bit dodgy looking, at http://autopagerize.net/ but their Extension for FireFox works and they mean well.
Enabling Autopagerize on Firefox
You can use GreaseMonkey and the AutoPagerize userscript if you like, but I use the AutoPagerize Firefox Extension from http://autopagerize.net/.
Enabling Autopagerize on Opera
Opera supports "User JavaScript" out of the box, so you can get their oAutoPagerize script, follow some directions and you're all set. It's a modification of the standard GreaseMonkey script and it will work with Safari and GreaseKit and Chrome, although I recommend the cleaner Chrome extension.
Enabling Autopagerize on Chrome
Chrome has a Chrome Extension called, logically enough, AutoPagerize for Chrome. It has the benefit of a small colored square in the address bar that will show you if the current page is enabled for paging and the current status.
I'm still looking into a reliable way to do this on IE, but you can start with the older GreaseMonkey for IE addon.
Enabling Autopagerize as a Web Site (Blog, etc) Publisher
Here's what it gets insane. Like "horribly gross and this will never scale" insane. There's two ways. If there are children in the room who design for the web, please ask them to leave.
First, you can go to this online database of sites http://wedata.net/databases/AutoPagerize/items and add your site along with some *cough* regular expressions and XPath expressions that describe where the next page to retrieve is and what to append it to. Wow, Regular Expressions AND XPath? What, no "select * from authors"? And a centralized database. Good times.
Well, my record (and most DasBlog sites) looks like:
pageElement: id("blog-posts")
url: ^http://www\.hanselman\.com/
nextLink: //div[@class="previous-posts"]/a
It basically says, you can find the next link at the anchor after the div with the class "previous posts" and you can append it to the element with the id of "blog-posts."
So this is gross.
Second option, and more ideally, I'd say, is this microformat. I'll actually copy/paste the microformat from the GreaseMonkey script itself as it says it all:
var MICROFORMAT = {
url: '.*',
nextLink: '//a[@rel="next"] | //link[@rel="next"]',
insertBefore: '//*[contains(@class, "autopagerize_insert_before")]',
pageElement: '//*[contains(@class, "autopagerize_page_element")]',
}
It says, find either an anchor like <a href="..." rel="next"> or a link in the head like <link rel="next" href="..."> then retrieve the page. Take the element with class "autopagerize_page_element" and append it to the element with "autopagerize_insert_before."
If your site/blog just adds a few classes and this rel, it'll be automatically setup to support autopagerize. I wanted to site my site like this but I hit a wall in the extensibility of DasBlog, the blog engine I run. This would be a small change to DasBlog, but it would mean a new version.
Of course, no browser supports this out of the box yet. Opera does offer a similar feature called "Fast Forward" that extends spacebar scrolling (in all browsers you can just press the spacebar to scroll down a page) such that it will navigate to the next page when you hit the bottom. Per Opera's KB:
Fast Forward tries to analyze a page and looks for links that will take you to the next page, for example after a search with Google with several pages of search results. It looks for certain patterns that indicate a "next" link, or uses "<link rel="next">" if it is defined in the page.
Unfortunately Opera analyzes my page and gets it wrong, selecting, oddly enough, an image as the next page to go to. This would likely be solved if I added a <link rel="next"> to my page's head, although again, I'd have to do this dynamically.
As an aside, notice this comment from Opera on their KB...
Please note that Fast Forward does not use any external services to determine the next page. It only looks at the current page and tries to find things that indicate that there is a "next" page. It does not look it up from an external server or contact any site to get this info.
This means they, too, realize that an external service is folly and the only way for this to work going forward is via microformats. I fervently agree.
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
Anybody know if it would be possible to adjust the script to make it still get the thumbnails? There is an alternative extension called AutoPager which it works on but it's not implemented as well as this one.
The other problem I can see is that currently the concept of infinite scrolling has to start from one end,, and not in the middle.
Don't forget that people WANT to deep link to a specific page of results.
That said, these are all design issues that can be solved by just thinking about the problem some more.
this solution will unify the way ASP.NET web sites manipulate Infinite Scroll
Nice, kinda reminds me of rockscroll you introduced to us to add on to Visual Studio.
Hanselminutes Fan,
Catto
I have had fun making a few user scripts mainly for resizing video on TED.com or other sites, and it really does make you wish there were similar features available in Internet Explorer.
Bookmarking will get messy quickly and the context of where you are on a page (in case you want to jump back to the top for some reason) will quickly get lost.
Apply carefully. There be dragons ahead.
However, something I couldn't get to work well was as follows: infinite scroll UP as well as down if landing in the middle of the infinite page. The appending of content below the viewport was fine - it didn't affect the viewport (the scrollbar would just extend) . However, adding a page ABOVE the viewport caused the viewport to be shifted down. I tried using jquery animated show together with a scroll but I coudn't get them properly synced so the pages wiggled as the new content appeared..
...which is of course, a moniker for completely useless. Could we please dump the "cool" and start delivering something actually useful?
Leave the "cool" to James Dean and Marlin Brando - ya know, nice guys, but both dead and buried.
Nice post, but this infinite scroll feature conflict with the fact that you have finite RAM, and this may cause that your computer crash, specially if the content size is too large !!!
Mostafa Nageeb
Comments are closed.
I think the "infinite scroll" feature is one of the most refreshing ways to provide paged data in a web UI. One of those things where I thought "why on earth isn't it used EVERYWHERE?".
By the way... really, why? Why only in Bing image search, and not in the standard Web search? :-)
Anyway, I'd love to see it also on Facebook Walls, or on twitter, where you don't get the "Next" set of data, but you have to scroll for "Older" items.