Object Recognition Resarch

The appearance of an object is composed of local structure. This local structure can be described and characterized by a vector of local features measured by local operators such as Gaussian derivatives or Gabor filters. We propose a a technique where appearances of objects are represented by the joint statistics of such local neighborhood operators. As such, this represents a new class of appearance based techniques for computer vision. Based on joint statistics, we have developed for the identification of multiple objects at arbitrary positions and orientations in a cluttered scene. Experiments show that these techniques can identify over 100 objects in the presence of major occlusions. Most remarkably, the techniques have low complexity and therefore run in real-time.

We therefore propose a framework for the statistical representation of the appearance of arbitrary 3D objects. This representation consists of a probability density function or joint statistics of local appearance as measured by a vector of robust local shape descriptors. The object representations are acquired automatically (learned) from sample images. Multidimensional histograms are introduced as a practical and reliable means for the approximation of the probability density function for local appearance. An important result of this paper is that the representation based on joint statistics of local neighborhood operators provides a reliable means for the representation and recognition of large sets of objects (over 100 objects) at arbitrary 3D positions and orientations in cluttered scenes.

Three different recognition algorithms are proposed within this framework and evaluated experimentally. The first algorithm compares the probability distribution of local neighborhood operators of a test image to the distributions of learned objects. Recognition is achieved by applying statistical divergence measurements which can be seen as a generalization of the color indexing scheme of Swain and Ballard [1991]. The second recognition algorithm calculates probabilities for the presence of objects based on a small number of vectors of local neighborhood operators. The experiments demonstrate that in the typical case, a small number of vectors is sufficient to obtain the correct object hypothesis from a database of 100 objects. In particular, experimental results show the robustness of the approach to partial occlusion. The most remarkable property of the the algorithm is that it relies on neither the calculation of correspondence nor figure ground segmentation of the object in the scene.

The second algorithm is extended to recognize multiple objects in cluttered scenes by using local appearance hashing. The capacity of the algorithm to recognize objects in cluttered scenes without relying on the calculation of correspondence is demonstrated experimentally. Due to its low complexity this algorithms runs on a standard Silicon Graphics O2-machine at 10Hz using the OpenGL-extension for real-time convolution of images.

It has been shown that the segmentation problem has exponential complexity in the size of the image considering no knowledge about the scene and in particular assuming no knowledge about which objects might be in the scene. However, the task-oriented visual search as e.g. in the case of segmenting objects knowing which objects are in the scene, has only linear complexity. Our probabilistic algorithm and its extension calculate object hypotheses with linear complexity (in the number of used image measurements and number of objects). This low complexity is mostly due to the fact that no correspondence and no segmentation are calculated. In that sense,we propose algorithms with linear complexity in order to obtain object hypotheses which can be used subsequently by a segmentation algorithm with linear complexity.

Probably the most comprehensive publication is the paper in IJCV - International Journal of Computer Vision. 36(1), p 31-50, January 2000 . If you want to read more about details the PhD thesis of Bernt Schiele is recommended.

Publications:

• Recognition without Correspondence using Multidimensional Receptive Field Histograms. Bernt Schiele and James L. Crowley. IJCV - International Journal of Computer Vision. 36(1), p 31-50, January 2000
• Probabilistic Object Recognition and Localization. Bernt Schiele and Alex Pentland. In ICCV'99 International Conference on Computer Vision.
• An Interactive Computer Vision System - DyPERS: Dynamic Personal Enhanced Reality System. Bernt Schiele, Nuria Oliver, Tony Jebara and Alex Pentland. In ICVS'99 International Conference on Vision Systems.
• Comprehensive Colour Image Normalization Graham D. Finlayson, Bernt Schiele and James L. Crowley. In ECCV'98 Fifth European Conference on Computer Vision.
• Using colour for image indexing Graham D. Finlayson, Bernt Schiele and James L. Crowley. In The Challenge of Image Retrieval.
• Transinformation for Active Object Recognition Bernt Schiele and James L. Crowley. In ICCV'98, International Conference on Computer Vision, Bombay, India, January 1998
• Object Recognition using Mutidimensional Receptive Field Histograms Bernt Schiele. PhD thesis, Institut National Polytechnique de Grenoble, english translation of the french thesis, July 1997.
• Transinformation of Object Recognition and its Application to Viewpoint Planning Bernt Schiele and James L. Crowley. In Robotics and Autonomous Systems, Vol 21, No 1, July 1997.
• Where to look next and what to look for. Bernt Schiele and James L. Crowley. In IROS'96, Intelligent Robots and Systems, Osaka, Japan, December 1996.
• Probabilistic Object Recognition Using Multidimensional Receptive Field Histograms. Bernt Schiele and James L. Crowley. In ICPR'96, International Conference on Pattern Recognition, Vienna, Austria, August 1996.
• Object Recognition Using Multidimensional Receptive Field Histograms. Bernt Schiele and James L. Crowley. In ECCV'96, Fourth European Conference on Computer Vision, Cambridge, UK, April 1996.
• The Robustness of Object Recognition to Rotation Using Multidimensional Receptive Field Histograms Bernt Schiele and James L. Crowley. Presented at Rosenon Worshop on Computational Vision, July 1995.