Hostname: page-component-848d4c4894-4rdrl Total loading time: 0 Render date: 2024-07-06T15:55:48.711Z Has data issue: false hasContentIssue false
  • English
  • Français

Geometric and Photometric Data Fusion in Non-Rigid Shape Analysis

Published online by Cambridge University Press:  28 May 2015

Artiom Kovnatsky ,
Dan Raviv ,
Michael M. Bronstein ,
Alexander M. Bronstein  and
Ron Kimmel
Artiom Kovnatsky
Affiliation:
Institute of Computational Science, Faculty of Informatics, Università della Svizzera Italiana, Lugano, Switzerland
Dan Raviv
Affiliation:
Technion - Israel Institute of Technology, Computer Science Department, Haifa, Israel
Michael M. Bronstein*
Affiliation:
Institute of Computational Science, Faculty of Informatics, Università della Svizzera Italiana, Lugano, Switzerland
Alexander M. Bronstein
Affiliation:
School of Electrical Engineering, Tel Aviv University, Israel
Ron Kimmel
Affiliation:
Technion - Israel Institute of Technology, Computer Science Department, Haifa, Israel
*
Corresponding author.Email address:michael.bronstein@gmail.com
Get access

Abstract

In this paper, we explore the use of the diffusion geometry framework for the fusion of geometric and photometric information in local and global shape descriptors. Our construction is based on the definition of a diffusion process on the shape manifold embedded into a high-dimensional space where the embedding coordinates represent the photometric information. Experimental results show that such data fusion is useful in coping with different challenges of shape analysis where pure geometric and pure photometric methods fail.

Keywords

65M1078A48Laplace-Beltrami operatordiffusion equationheat kernel descriptors3D shape retrievaldeformation invariance
Type
Research Article
Information
Numerical Mathematics: Theory, Methods and Applications , Volume 6 , Issue 1 , February 2013 , pp. 199 - 222
Copyright
Copyright © Global Science Press Limited 2013

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

[1] Aflalo, J., Raviv, D. and Kimmel, R., Scale invariant geometry for non-rigid shapes, Technical Report CIS-2011-02, Dept. of Computer Science, Technion, Israel, 2011.Google Scholar
[2] Amores, J., Sebe, N. and Radeva, P., Context-based object-class recognition and retrieval by generalized correlograms, Trans. PAMI, 29(10) (2007), pp. 1818–1833.Google Scholar
[3] Arya, S., Mount, D. M., Netanyahu, N. S., Silverman, R. and Wu, A. Y., An optimal algorithm for approximate nearest neighbor searching, J. Acm, 45 (1998), pp. 891–923.Google Scholar
[4] Assfalg, J., Bertini, M., Del, A. BIMBO and Pala, P., Content-based retrieval of 3-D objects using spin image signatures, Trans. Multimedia, 9(3) (2007), pp. 589–599.CrossRefGoogle Scholar
[5] Aubry, M., Schlickewei, U. and Cremers, D., The wave kernel signature-a quantum mechanical approach to shape analyis, In Proc. CVPR, 2011.Google Scholar
[6] Belkin, M. and Niyogi, P., Laplacian eigenmaps for dimensionality reduction and data representation, Neural Computation, 13 (2003), pp. 1373–1396.Google Scholar
[7] Belkin, M., Sun, J. and Wang, Y., Discrete Laplace operator on meshed surfaces, In Proc. SCG, pp. 278-287, 2008.Google Scholar
[8] Belkin, M., Sun, J. and Wang, Y, Constructing Laplace operator from point clouds in ∝d, In Proc. SODA, pages 1031–1040, 2009.Google Scholar
[9] Bérard, P., Besson, G. and Gallot, S., Embedding Riemannian manifolds by their heat kernel, GAFA, 4(4) (1994), pp. 373–398.Google Scholar
[10] Bronstein, A. M., Spectral descriptors of deformable shapes, Trans. PAMI, 2011, submitted.Google Scholar
[11] Bronstein, A. M., Bronstein, M. M., Castellani, U., Dubrovina, A., Guibas, L. J., Horaud, R. P., Kimmel, R., Knossow, D., Von Lavante, E., Mateus, D., Ovsjanikov, M. and Sharma, A., SHREC 2010: robust correspondence benchmark, In Proc. 3DOR, 2010.Google Scholar
[12] Bronstein, A. M., Bronstein, M. M., Castellani, U., Falcidieno, B., Fusiello, A., Godil, A., Guibas, L. J., Kokkinos, I., Lian, Z., Ovsjanikov, M., Patane, G., Spagnuolo, M. and Sun, J., SHREC 2010: robust large-scale shape retrieval benchmark, In Proc. 3DOR, 2010.Google Scholar
[13] Bronstein, A. M., Bronstein, M. M. and Kimmel, R., Efficient computation of isometry-invariant distances between surfaces, SIAM J. Sci. Comput., 28(5) (2006), pp. 1812–1836.Google Scholar
[14] Bronstein, A. M., Bronstein, M. M. and Kimmel, R., Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching, PNAS, 103(5) (2006), pp. 1168–1172.Google Scholar
[15] Bronstein, A. M., Bronstein, M. M., Kimmel, R., Mahmoudi, M. and Sapiro, G., A Gromov- Hausdorffframework with diffusion geometry for topologically-robust non-rigid shape matching, IJCV, 89(2-3) (2010), pp. 266–286.Google Scholar
[16] Bronstein, A. M., Bronstein, M. M., Ovsjanikov, M. and Guibas, L. J., Shape google: a computer vision approach to invariant shape retrieval, In Proc. NORDIA, 2009.Google Scholar
[17] Bronstein, M. M. and Bronstein, A. M., Shape recognition with spectral distances, Trans. PAMI, 33(5) (2011), 106.Google Scholar
[18] Bronstein, M. M. and Kokkinos, I., Scale-invariant heat kernel signatures for non-rigid shape recognition, In Proc. CVPR, 2010.Google Scholar
[19] Chum, O., Philbin, J., Sivic, J., Isard, M. and Zisserman, A., Total recall: automatic query expansion with a generative feature model for object retrieval, In Proc. ICCV, 2007.Google Scholar
[20] Coifman, R. R. and Lafon, S., Diffusion maps, Appl. Compu. Harmonic Anal., 21(1) (2006), pp. 5–30.Google Scholar
[21] Digne, J., Morel, J. M., Audfray, N. and Mehdi-Souzani, C., The level set tree on meshes, In Proc. 3DPVT, 2010.Google Scholar
[22] Elad, A. and Kimmel, R., On bending invariant signatures for surfaces, Trans. PAMI, 25(10) (2003), pp. 1285–1311.Google Scholar
[23] Gebal, K., Bærentzen, J. A., Aanæs, H. and Larsen, R., Shape analysis using the auto diffusion function, In Computer Graphics Forum, 28 (2009), pp. 1405–1413.Google Scholar
[24] Gelfand, N., Mitra, N. J., Guibas, L. J. and Pottmann, H., Robust global registration, In Proc. SGP, 2005.Google Scholar
[25] Hamza, A. B. and Krim, H., Geodesic object representation and recognition, In Proc. DGCI, pages 378–387, 2003.Google Scholar
[26] Hochbaum, D. S. and Shmoys, D. B., A best possible heuristic for the k-center problem, Math. Operations Research, 10(2) (1985), pp. 180–184.Google Scholar
[27] Jain, A. K., Fundamentals of Digital Image Processing, Prentice-Hall Information and System Sciences Series, Prentice Hall, 1989.Google Scholar
[28] Jones, P. W., Maggioni, M. and Schul, R., Manifold parametrizations by eigenfunctions of the Laplacian and heat kernels, PNAS, 105(6) (2008), 180.Google Scholar
[29] Kazhdan, M., Funkhouser, T. and Rusinkiewicz, S., Rotation invariant spherical harmonic representation of 3D shape descriptors, In Proc. SGP, pages 156–164, 2003.Google Scholar
[30] Kazhdan, M. M., Funkhouser, T. A. and Rusinkiewicz, S., Shape matching and anisotropy, TOG, 23(3) (2004), pp. 623–62.Google Scholar
[31] Kimmel, R., Malladi, R. and Sochen, N., Images as embedded maps and minimal surfaces: movies, color, texture and volumetric medical images, IJCV, 39(2) (2000), pp. 111–129.Google Scholar
[32] Kovnatsky, A., Bronstein, A. M., Bronstein, M. M. and Kimmel, R., Photometric heat kernel signatures, In Proc. SSVM, 2011.Google Scholar
[33] B. LÉVY, Laplace-Beltrami eigenfunctions towards an algorithm that “understands” geometry, In Proc. SMI, 2006.Google Scholar
[34] Ling, H. and Jacobs, D. W., Deformation invariant image matching, In Proc. ICCV, pages 1466– 1473, 2005.Google Scholar
[35] Litman, R., Bronstein, A. M. and Bronstein, M. M., Diffusion-geometric maximally stable component detection in deformable shapes, CG, 35(3) (2011), pp. 549–560.Google Scholar
[36] Lowe, D., Distinctive image features from scale-invariant keypoint, IJCV, 2004.Google Scholar
[37] Mahmoudi, M. and Sapiro, G., Three-dimensional point cloud recognition via distributions of geometric distances, Graphical Models, 71(1) (2009), pp. 22–31.CrossRefGoogle Scholar
[38] Matas, J., Chum, O., Urban, M. and Pajdla, T., Robust wide-baseline stereo from maximally stable extremal regions, Image Vision Comput., 22(10) (2004), pp. 761–767.Google Scholar
[39] F. MÉMOLI, Spectral Gromov-Wasserstein distances for shape matching, In Proc. NORDIA, 2009.Google Scholar
[40] Mémoli, F. and Sapiro, G., A theoretical and computational framework for isometry invariant recognition of point cloud data, IMA preprint series 1980, University of Minnesota, 2004.Google Scholar
[41] Mémoli, F. and Sapiro, G., A theoretical and computational framework for isometry invariant recognition of point cloud data, Found. Comput. Math., 5 (2005), pp. 313–346.Google Scholar
[42] Ohbuchi, R., Osada, K., Furuya, T. and Banno, T., Salient local visual features for shape-based 3D model retrieval, In Proc. SMI, 2008.Google Scholar
[43] Osada, R., Funkhouser, T., Chazelle, B. and Dobkin, D., Shape distributions, TOG, 21(4) (2002), pp. 807–832.Google Scholar
[44] Ovsjanikov, M., Sun, J. and Guibas, L. J., Global intrinsic symmetries of shapes, Comput. Graphics Forum, 27(5) (2008), pp. 1341–1348.Google Scholar
[45] Pan, X., Zhang, Y., Zhang, S. and Ye, X., Radius-normal histogram and hybrid strategy for 3D shape retrieval, In Proc. SMI, 2005.Google Scholar
[46] Raviv, D., Bronstein, A. M., Bronstein, M. M. and Kimmel, R., Full and partial symmetries of non-rigid shapes, IJCV, 89 (2009), pp. 18–39.Google Scholar
[47] Raviv, D., Bronstein, A. M., Bronstein, M. M., Kimmel, R. and Sochen, N., Affine-invariant geodesic geometry of deformable 3D shapes, Comput. Graphics, 35(3) (2011), pp. 692–697.Google Scholar
[48] Raviv, D., Bronstein, M. M., Bronstein, A. M. and Kimmel, R., Volumetric heat kernel signatures, 3DOR, 2010.Google Scholar
[49] Reuter, M., Wolter, F.-E. and Peinecke, N., Laplace-spectra as fingerprints for shape matching, In Proc. SPM, pages 101–106, 2005.Google Scholar
[50] Rustamov, R. M., Laplace-Beltrami eigenfunctions for deformation invariant shape representation, In Proc. SGP, pages 225-233, 2007.Google Scholar
[51] Sivic, J. and Zisserman, A., Video google: a text retrieval approach to object matching in videos, In Proc. CVPR, 2003.Google Scholar
[52] Skraba, P., Ovsjanikov, M., Chazal, F. and Guibas, L., Persistence-based segmentation of de-formable shapes, In Proc. NORDIA, pages 45–52, 2010.Google Scholar
[53] Sochen, N., On affine invariance in the beltrami framework for vision, In Proc. VLSM, 2001.Google Scholar
[54] Sun, J., Ovsjanikov, M. and Guibas, L., A Concise and provably informative multi-scale signature based on heat diffusion, Comput. Graphics Forum, 28(5) (2009), pp. 1383–1392.Google Scholar
[55] Sun, J., Ovsjanikov, M. and Guibas, L. J., A concise and provably informative multi-scale signature based on heat diffusion, In Proc. SGP, 2009.Google Scholar
[56] Thangudu, K., Practicality of Laplace Operator, Master’s thesis, The Ohio State University, Computer Science and Engineering Department, 2009.Google Scholar
[57] Thorstensen, N. and Keriven, R., Non-rigid shape matching using geometry and photometry, In Proc. CVPR, 2009.Google Scholar
[58] Toldo, R., Castellani, U. and Fusiello, A., Visual vocabulary signature for 3D object retrieval and partial matching, In Proc. 3DOR, 2009.Google Scholar
[59] Tomasi, C. and Manduchi, R., Bilateral fitering for gray and color images, In Proc. ICCV, pages 839–846, 1998.Google Scholar
[60] Vranic, D.V., Saupe, D. and Richter, J., Tools for 3D-object retrieval: Karhunen-Loeve transform and spherical harmonics, In MMSP, pages 293-298, 2001.Google Scholar
[61] Wardetzky, M., Mathur, S., Kälberer, F. and Grinspun, E., Discrete Laplace operators: no free lunch, In Proc. SGP, 2008.Google Scholar
[62] Wu, C., Clipp, B., Li, X., Frahm, J.-M. and Pollefeys, M., 3D model matching with viewpoint-invariant patches (VIP), In Proc. CVPR, 2008.Google Scholar
[63] Wyngaerd, J.V., Combining texture and shape for automatic crude patch registration, DIM, pages 179–186, 2003.Google Scholar
[64] Xu, G., Convergence of discrete Laplace-Beltrami operators over surfaces, Technical report, Institute of Computational Mathematics and Scientific/Engineering Computing, China, 2004.Google Scholar
[65] Yoon, K.-J., Prados, E. and Sturm, P., Joint estimation of shape and reflectance using multiple images with known illumination conditions, IJCV, 86(2-3) (2010), pp. 192–210.Google Scholar
[66] Zaharescu, A., Boyer, E. and Horaud, R. P., Transformesh: a topology-adaptive mesh-based approach to surface evolution, In Proc. ACCV, 2007.Google Scholar
[67] Zaharescu, A., Boyer, E., Varanasi, K. and Horaud, R, Surface feature detection and description with applications to mesh matching, In Proc. CVPR, 2009.Google Scholar