Mobile Image Webs: Finding and Using Structure in Large Image Collections

Introduction

The widespread availability of digital cameras and ubiquitous Internet access have facilitated the creation of massive image collections. These collections can be highly interconnected through implicit links between image pairs viewing the same or similar objects. We build graphs called Image Webs to represent such connections. In this project, our goal is to use image webs to provide useful services for mobile users including content based image retrieval, query expansion, image annotation and augmented reality.

Image Webs Overview

Image web can be used to discover the connections between images in the collection induced by shared objects. We can connect images that are visually different.
Connection through repeated objects

Connection through mobile objects

The Structure of Image Web

  • The idea of Image Webs is to interlink images through a variety of link types.
  • Image web is a graph where vertices are associated with parts of an image and edges represent relations between the regions.

Applications

Peer to Peer Image Webs

When two mobile users have their own image collections and Image Webs respectively, we can find similar images in two collections without sending all images to a central place. We utilize the computing power of the mobile devices to find the compact signature of each image and compute the distance between images. Multiple mobile users can share similar images with each other and merge their own Image Webs using peer-to-peer protocols.

Cooperative Image Annotation

With cooperative image annotation, we can label a small subset of existing or new images and have the label propagate to the rest of the collection through Image Webs. When a user upload new images and corresponding annotations to the Image Web, the labels are propagated in the Image Web using graph learning algorithm. When the mobile user add a new unlabeled image to the Image Web, we can transfer annotation from the labeled Image Web to this new image.

In a social network built using a distributed architecture such as PRPL, each user has an image collection and an Image Web. The user has access to the pictures shared by friends. By overlaying Image Web through these shared images, the annotations of images are propagated in the community.

Publications

Zixuan Wang, Omprakash Gnawali, Kyle Heath, and Leonidas Guibas, Collaborative Image Annotation Using Image Webs, In proceedings of the Army Science Conference (ASC 2010), November 2010. Acceptance Rate for Sensors and Information Processing Track - 20/93 (21.5%)
Last updated: November 11, 2010