Deep Zoom (Seadragon) Silverlight 2 MultiScaleImages at Mix
One of the cool things seen at Mix 08 today was the Deep Zoom technology. This was originally called Seadragon and came out of the Live Labs research.
You can see it live at the Hard Rock Cafe Memorabilia site. This Silverlight application includes a tiled montage of 512 many-megapixel images, some 10+megapixels, while some are stitched together to sizes upwards of 40-50 megapixels, it seems.
Remember the first 3D games like Sega's OutRun? They seemed 3D, but basically bitmaps were being scaled as you "drove forward" and each graphical asset, like a house, for example, would have a number of pre-rendered versions. Another example are .ico files that have 16x16, 32x32, etc, versions. As the viewer there is one image, but there's multiple resolutions depending on how far in you want to zoom. This is the basic idea behind "multi-scale images."
You can see examples of exceedingly large images, often called "giga-pixel" images all over the web. A great example of zooming using Flash is at Harlem 13-Gigapixels.com. The large Harlem image is 2,045 individual images of 12 megapixels each. The rendering when zooming in is a lot like Google Earth. You zoom into an image of a specific resolution then that first image transitions into a tile of a second image rendered another resolution. You can create these kinds of images with Panorama stitching software like AutoPano.net, and PanoramaFactory. I took a large 50+ megapixel panorama of Ngorongoro Crater in Tanzania that I think would make a good example for this kind of thing. I'll dig it up and make a Deep Zoom example project.
DeepZoom includes an optimized local Silverlight Renderer and XAML control that makes creating these fairly easy. Try it out over at the HardRock Memorabilia site and take specific notice of this guitar I've circled in the picture below.
Zoom in...
And more...into the screw knob in the lower left corner...
You can actually see a reflection of the bench and the camera and Scott Stanfield from Vertigo who's taking the shot of this guitar, reflected in the shiny knob. Vertigo wrote the Hardrock site and they also have a great Deep Zoom Gallery.
You can go download the Preview of the Deep Zoom Composer now. You can see its heritage when you make a new Project and it puts it in a folder called "Seadragon Projects." Here's the Deep Zoom Composer User Guide.
You place a number of pictures (or a large one) on the surface and export the collection. There will be a scene graph XML file created indicating location, size, z-order and aspect ratio of the images.
Then an output directory will be created with folders for each image and, depending on the relative largeness of the source images, a series of tiles generated a number of zoom levels. Here's an example directory structure:
See how that image at zoom level 11 has 20 different tiles? They've got to balance many factors to make it all seamless. They're considering how many tiles are needed for that resolution as it relates to a zoom level's need for memory, as well as considering a reasonable number of requests to the server. There's the extreme need for a good user-experience so you want the images to come fast and furious, be easily cached and quickly shown to give the perception of speed. You don't have to sweat any of this stuff, as the multi-resolution tiles stitched together automatically by the client rendered.
After you've exported a project into these files, you create a MultiScale image in XAML like this, referencing a bin file manifest that knows contains info about the directory structure and files:
<MultiScaleImage x:Name="deepZoomObject" Source="foo/info.bin"/>
This gets you the image rendered. To zoom a MultiScaleImage, you call ZoomAboutLogicalPoint:
public void ZoomAboutLogicalPoint(double zoomIncrementFactor, double zoomCenterLogicalX, double zoomCenterLogicalY)
But there's a decent amount of state management that needs to be done as well as deciding how you want things animated, whether or not your image contains sub-images, etc. In the morning I'll take a look at how to hook up JavaScript events like Mouse Wheel Scrolls and do some zooming and panning. Hopefully Vertigo will release their hard-won source code for the canonical example to the community. It's pretty slick.
About Scott
Scott Hanselman is a former professor, former Chief Architect in finance, now speaker, consultant, father, diabetic, and Microsoft employee. He is a failed stand-up comic, a cornrower, and a book author.
About Newsletter
which has been around for a few years.
I found all the methods and properties but no documentation at all. Anyone found a solution? Such a tease.
The best so far is using the keyboard here.
John.
We've used Zoomify previously, courtesy of the free converter available on the Zoomify website, and it works fine. But, Zoomify's free converter is limited in several ways: there's no control over the degree of JPEG compression, it doesn't wrap up the resulting image pieces and Flash-based viewer in the html code necessary to post a Zoomified picture on the Web (instead, a code template and instructions for manually editing it are included) and the viewer itself is a bit clunky: the control bar overlays the picture and sometimes too easily blends into image content, it's possible to scroll past the edges of the picture and the loading of new picture segments, while fast, lacks the smoothness we're accustomed to from Flash content these days.
None of these things are deal-stoppers, but the Zoomify experience has been just clumsy enough to limit our desire to use it regularly. The paid-for Zoomify converter that removes some of these limitations is an add-on to a full-fledged Macromedia/Adobe Flash application like Flash Professional 8, which itself is more expensive than we've been able to justify to date.
Seems like there's a new gigapixel "image war" on the 'net. It'll be interesting what Flash comes up with (presumably very soon) around big images. I've seen at least 6 different flash gigapixel image viewers that I assume are custom code.
but what about photosynth? is it coming out any time soon
anyhow this is good news, I already have some uses for it ^^"
So I now have all my photos rendering, the mouse working to pan nicely at all zoom levels but i'm yet to be able to hook up the mouse scroll wheel, only able to use +,- key currently. I think I need to capture the scroll event in javascript - anyone cracked it yet?
John.
Regarding the mouse wheel stuff: we haven't decided when/if/how we're going to release it. We've been so focused on just getting this launched and the keynote, we haven't had time to think about it.
We've used it a lot in our work with virtual microscopy, but unfortunately it has been kept primarily inhouse or at least in-market (mainly because of the subject matter)
A few years ago Aperio released something remarkably similar to the Harlem 13-Gigapixels.com interface for their online gallery (Example http://images2.aperio.com/Dev.%20Incisor.svs/view.apml ), but even that was only really seen by like-minded souls.
SeaDragon did well to see the wider uses, and did a good job in smoothing out the various issues with the techniques, and now it's been at MIX with downloadable tools? Well it's going to be everywhere :)
Here is my attempt at humor.
It is also nice that they finally provide a tool where we can see both Scott's forehead and his facial features in the same shot without scrolling my browser or downloading then opening up paint.net to crop some of that thing out :)
http://www.soulsolutions.com.au/Blog/tabid/73/EntryID/394/Default.aspx
It includes the mouse wheel, keyboard controls and catching a few bugs. Download the code and put in your images.
See the sample of high res images (54mb) from Thailand as an example (zoom in on a small monkey):
thailand.soulclients.com
John.
I'm sure MS has a VirtualEarth proof-of-concept using SeaDragon out there some where. Let's see it! :)
I was thinking the same, it would suit a maps programme so well. I love the way the extra data seems to melt into view. I've made my own demo using a photo of the office of the design studio I work for. Couldn't have done it without help from this page. Thanks!
Just a few fact checking, mind you?
- SeaDragon is not a Microsoft research project, it's the result of the acquisition of an external company.
- What SeaDragon does technically speaking? it's a client viewer for a simple variant of JPEG 2000 LOD files. LOD = Level Of Details. This is a technique coming from Fourier/Wavelet transforms where you can interpolate a picture of size d, between two predefined pictures of size d1 and d2. What makes this technique special is that it tries to control the distortion and other artifacts resulting from the scaling. In a way, you can call it a better JPEG encoding. (hence JPEG 2000, which is patent encumbered unfortunately)
- This is actually pretty standard modern mathematics, nothing to write home about.
- No one should read your post thinking that this "breakthrough technology" creates picture details from nothing. Shannon rules of entropy explain why this is not possible, and that's pretty obvious anyway that you can't do that. So it's always a bit disingenuous, although sexy I admit, to zoom in a picture in a way that may make the audience believe that details are generated as it zooms in. A more genuine showcase is to zoom out of a picture, because that's what better explains what the technology does really, ie remove details under controlled distortion.
Social Helix - the future of event networking
*controls - use the scroll wheel, the zoom bar, or circle your mouse clockwise or counterclockwise to zoom in and out.
most of these site load the images very slowly. i was actually very impressed with the hard rock implementation. I can wait until we can afford our next speed boost.
Comments are closed.
Maybe it's because I've installed Vista SP1, but according to a thread on Channel 9 I'm not the only one having this problem.