Goal Search engine for 3D models Challenges Simple query methods Robust shape matching methods

A decade ago Draw a picture Copy a picture Photograph a picture Today Search from the web 3D model?

Easy to construct New scanners Interactive tools Fast, inexpensive graphics hardware Widespread distribution

How do we construct them? How do we find them? A city scene Cars Street lamps Stop signs Research in retrieval, matching, recognition, classification

Keywords? Not annotated Annotated with inspecific key words Too common Not known Shape-based query Distinguish similar objects A picture is worth a thousand words

Shape descriptors Spherical harmonics descriptors User interfaces 3D sketching 2D sketching Text Interactive refinement

2D hard to extend Registration, recognition, matching Time consuming Local, sensitive Statistical properties Not discriminating enough Specific data types CAD parts, protein molecules, cultural artifacts

Indexed a large repository Supported 2D and 3D sketching Studied interaction between text and shape

Crawling 17,834 VRML 2,873 commercial Indexing Descriptive Concise Efficient to compute Robust to model degeneracy invariant to rotations Query Text keywords 2D sketching 3D sketching Model matching Iterative refinement Matching 16 results < 0.25 seconds

Two types File name Find similar shape Desirable descriptor Quick to compute Concise to store Easy to index Invariant under similarity transformations Insensitive to noise and small extra features Independent of 3D object representation, tessellation, or genus Robust to arbitrary topological degeneracies Discriminating of shape differences at many scales

Most high-level shape representations Require a consistent model of the objects boundary and interior Other shape representations Require a priori registration into a canonical coordinate system Statistical shape descriptors Not discriminating enough

Euclidean distance K best matches: k nearest neighbours Search in 1D space first Compute true distance

3D sketch Teddy + matching Hard to learn Extrusion Cuts Rotate Blobby objects: genus zero

2D sketch Pixel paint + match 2D projections Properties Easy Less info: multiple projections Challenge Match 2D sketches to 3D objects

Algorithm Computer matching score Each pair of m sketches versus 13 2D views Minimal sum of m pairwise sketch-to-thumbnail scores No thumbnail can be matched to >1 sketch Advantages Any sketched view is with 22.5o of a sample view Some aligned with axes Label side, front, and top

Fourier Descriptors

Representative document Filename, web text, labels inside model file Removing common words Stem text Add synonyms Matching tool: Bow toolkit

Example 1 First query: text Second query: text + shape Example 2

Algorithm for finding K top matches Find the best M matches for each mode Mean-normalize them (mean 0, var 1) Return K models with the highest average normalized scores Currently K=16, M=128

Query interfaces: text, 2D/3D sketches, 3D models New descriptor Shape + text = useful A large repository of 3D models