August 8, 2006: We previously wrote about Microsoft’s interesting new Group Shot program, available free here. Today, I learned about Photosynth, a sophisticated tool that combines shots of the same area into a 3D virtual landscape. It’s all very high-concept, so I’ll let the good people at Microsoft describe it:

Photosynth takes a large collection of photos of a place or object, analyzes them for similarities, and displays them in a reconstructed 3-Dimensional space.

Your brain knows that your eyes are about two inches apart. But when Photosynth does its magic, it doesn’t know where the cameras were, or which way they were pointing. Fortunately, when there are many cameras, and many features in common, the algorithms behind Photosynth can figure out not only where the features are in 3D, but where all of the cameras would have to have been, and which way they were aimed, consistent with the features they ”saw”.

With Photosynth you can:

• Walk or fly through a scene to see photos from any angle.
  Seamlessly zoom in or out of a photograph whether it’s megapixels or gigapixels in size.
  See where pictures were taken in relation to one another.
  Find similar photos to the one you’re currently viewing.
  Explore a custom tour.
  Send a collection to a friend.A Photosynth experience begins with nothing more than a bunch of digital photos. They might all have been taken by you, or they might be a mixture of images from many different cameras, shooting conditions, dates, times of day, resolutions, and so on

Each photo is processed by computer vision algorithms to extract hundreds of distinctive features, like the corner of a window frame or a doorhandle. Then, photos that share features are linked together in a web. When a feature’s found in multiple images, its 3D position can be calculated. It’s similar to depth perception—what your brain does to perceive the 3D positions of things in your field of view based on their images in both of your eyes. Photosynth’s 3D model is just the cloud of points showing where those features are in space.

The Photosynth client shows you the 3D pointcloud, but more importantly, it also shows you the original pictures overlaid on the model. Imagine a slide projector placed at each camera position, aimed where the camera was aiming, and projecting the picture that camera took. A screen is placed in the 3D environment at an appropriate distance from the projector. As you move around in the Photosynth environment, projectors turn on and off, giving you a changing perspective on a world built entirely out of the original photos.

How cool does this sound? The program isn’t out yet, but Microsoft has some previews and updates here.

This entry was posted on Wednesday, August 9th, 2006 at 4:30 am and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.