Skip to main content Accessibility help

Geometric and Photometric Data Fusion in Non-Rigid Shape Analysis

  • Artiom Kovnatsky (a1), Dan Raviv (a2), Michael M. Bronstein (a1), Alexander M. Bronstein (a3) and Ron Kimmel (a2)...


In this paper, we explore the use of the diffusion geometry framework for the fusion of geometric and photometric information in local and global shape descriptors. Our construction is based on the definition of a diffusion process on the shape manifold embedded into a high-dimensional space where the embedding coordinates represent the photometric information. Experimental results show that such data fusion is useful in coping with different challenges of shape analysis where pure geometric and pure photometric methods fail.


Corresponding author

Corresponding author. Email address:


Hide All
[1] Aflalo, J., Raviv, D. and Kimmel, R., Scale invariant geometry for non-rigid shapes, Technical Report CIS-2011-02, Dept. of Computer Science, Technion, Israel, 2011.
[2] Amores, J., Sebe, N. and Radeva, P., Context-based object-class recognition and retrieval by generalized correlograms, Trans. PAMI, 29(10) (2007), pp. 18181833.
[3] Arya, S., Mount, D. M., Netanyahu, N. S., Silverman, R. and Wu, A. Y., An optimal algorithm for approximate nearest neighbor searching, J. Acm, 45 (1998), pp. 891923.
[4] Assfalg, J., Bertini, M., Del, A. BIMBO and Pala, P., Content-based retrieval of 3-D objects using spin image signatures, Trans. Multimedia, 9(3) (2007), pp. 589599.
[5] Aubry, M., Schlickewei, U. and Cremers, D., The wave kernel signature-a quantum mechanical approach to shape analyis, In Proc. CVPR, 2011.
[6] Belkin, M. and Niyogi, P., Laplacian eigenmaps for dimensionality reduction and data representation, Neural Computation, 13 (2003), pp. 13731396.
[7] Belkin, M., Sun, J. and Wang, Y., Discrete Laplace operator on meshed surfaces, In Proc. SCG, pp. 278-287, 2008.
[8] Belkin, M., Sun, J. and Wang, Y, Constructing Laplace operator from point clouds in ∝d, In Proc. SODA, pages 10311040, 2009.
[9] Bérard, P., Besson, G. and Gallot, S., Embedding Riemannian manifolds by their heat kernel, GAFA, 4(4) (1994), pp. 373398.
[10] Bronstein, A. M., Spectral descriptors of deformable shapes, Trans. PAMI, 2011, submitted.
[11] Bronstein, A. M., Bronstein, M. M., Castellani, U., Dubrovina, A., Guibas, L. J., Horaud, R. P., Kimmel, R., Knossow, D., Von Lavante, E., Mateus, D., Ovsjanikov, M. and Sharma, A., SHREC 2010: robust correspondence benchmark, In Proc. 3DOR, 2010.
[12] Bronstein, A. M., Bronstein, M. M., Castellani, U., Falcidieno, B., Fusiello, A., Godil, A., Guibas, L. J., Kokkinos, I., Lian, Z., Ovsjanikov, M., Patane, G., Spagnuolo, M. and Sun, J., SHREC 2010: robust large-scale shape retrieval benchmark, In Proc. 3DOR, 2010.
[13] Bronstein, A. M., Bronstein, M. M. and Kimmel, R., Efficient computation of isometry-invariant distances between surfaces, SIAM J. Sci. Comput., 28(5) (2006), pp. 18121836.
[14] Bronstein, A. M., Bronstein, M. M. and Kimmel, R., Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching, PNAS, 103(5) (2006), pp. 11681172.
[15] Bronstein, A. M., Bronstein, M. M., Kimmel, R., Mahmoudi, M. and Sapiro, G., A Gromov- Hausdorffframework with diffusion geometry for topologically-robust non-rigid shape matching, IJCV, 89(2-3) (2010), pp. 266286.
[16] Bronstein, A. M., Bronstein, M. M., Ovsjanikov, M. and Guibas, L. J., Shape google: a computer vision approach to invariant shape retrieval, In Proc. NORDIA, 2009.
[17] Bronstein, M. M. and Bronstein, A. M., Shape recognition with spectral distances, Trans. PAMI, 33(5) (2011), 106.
[18] Bronstein, M. M. and Kokkinos, I., Scale-invariant heat kernel signatures for non-rigid shape recognition, In Proc. CVPR, 2010.
[19] Chum, O., Philbin, J., Sivic, J., Isard, M. and Zisserman, A., Total recall: automatic query expansion with a generative feature model for object retrieval, In Proc. ICCV, 2007.
[20] Coifman, R. R. and Lafon, S., Diffusion maps, Appl. Compu. Harmonic Anal., 21(1) (2006), pp. 530.
[21] Digne, J., Morel, J. M., Audfray, N. and Mehdi-Souzani, C., The level set tree on meshes, In Proc. 3DPVT, 2010.
[22] Elad, A. and Kimmel, R., On bending invariant signatures for surfaces, Trans. PAMI, 25(10) (2003), pp. 12851311.
[23] Gebal, K., Bærentzen, J. A., Aanæs, H. and Larsen, R., Shape analysis using the auto diffusion function, In Computer Graphics Forum, 28 (2009), pp. 1405–1413.
[24] Gelfand, N., Mitra, N. J., Guibas, L. J. and Pottmann, H., Robust global registration, In Proc. SGP, 2005.
[25] Hamza, A. B. and Krim, H., Geodesic object representation and recognition, In Proc. DGCI, pages 378–387, 2003.
[26] Hochbaum, D. S. and Shmoys, D. B., A best possible heuristic for the k-center problem, Math. Operations Research, 10(2) (1985), pp. 180–184.
[27] Jain, A. K., Fundamentals of Digital Image Processing, Prentice-Hall Information and System Sciences Series, Prentice Hall, 1989.
[28] Jones, P. W., Maggioni, M. and Schul, R., Manifold parametrizations by eigenfunctions of the Laplacian and heat kernels, PNAS, 105(6) (2008), 180.
[29] Kazhdan, M., Funkhouser, T. and Rusinkiewicz, S., Rotation invariant spherical harmonic representation of 3D shape descriptors, In Proc. SGP, pages 156–164, 2003.
[30] Kazhdan, M. M., Funkhouser, T. A. and Rusinkiewicz, S., Shape matching and anisotropy, TOG, 23(3) (2004), pp. 623–62.
[31] Kimmel, R., Malladi, R. and Sochen, N., Images as embedded maps and minimal surfaces: movies, color, texture and volumetric medical images, IJCV, 39(2) (2000), pp. 111–129.
[32] Kovnatsky, A., Bronstein, A. M., Bronstein, M. M. and Kimmel, R., Photometric heat kernel signatures, In Proc. SSVM, 2011.
[33] B. LÉVY, Laplace-Beltrami eigenfunctions towards an algorithm that “understands” geometry, In Proc. SMI, 2006.
[34] Ling, H. and Jacobs, D. W., Deformation invariant image matching, In Proc. ICCV, pages 1466– 1473, 2005.
[35] Litman, R., Bronstein, A. M. and Bronstein, M. M., Diffusion-geometric maximally stable component detection in deformable shapes, CG, 35(3) (2011), pp. 549–560.
[36] Lowe, D., Distinctive image features from scale-invariant keypoint, IJCV, 2004.
[37] Mahmoudi, M. and Sapiro, G., Three-dimensional point cloud recognition via distributions of geometric distances, Graphical Models, 71(1) (2009), pp. 22–31.
[38] Matas, J., Chum, O., Urban, M. and Pajdla, T., Robust wide-baseline stereo from maximally stable extremal regions, Image Vision Comput., 22(10) (2004), pp. 761–767.
[39] F. MÉMOLI, Spectral Gromov-Wasserstein distances for shape matching, In Proc. NORDIA, 2009.
[40] Mémoli, F. and Sapiro, G., A theoretical and computational framework for isometry invariant recognition of point cloud data, IMA preprint series 1980, University of Minnesota, 2004.
[41] Mémoli, F. and Sapiro, G., A theoretical and computational framework for isometry invariant recognition of point cloud data, Found. Comput. Math., 5 (2005), pp. 313–346.
[42] Ohbuchi, R., Osada, K., Furuya, T. and Banno, T., Salient local visual features for shape-based 3D model retrieval, In Proc. SMI, 2008.
[43] Osada, R., Funkhouser, T., Chazelle, B. and Dobkin, D., Shape distributions, TOG, 21(4) (2002), pp. 807–832.
[44] Ovsjanikov, M., Sun, J. and Guibas, L. J., Global intrinsic symmetries of shapes, Comput. Graphics Forum, 27(5) (2008), pp. 1341–1348.
[45] Pan, X., Zhang, Y., Zhang, S. and Ye, X., Radius-normal histogram and hybrid strategy for 3D shape retrieval, In Proc. SMI, 2005.
[46] Raviv, D., Bronstein, A. M., Bronstein, M. M. and Kimmel, R., Full and partial symmetries of non-rigid shapes, IJCV, 89 (2009), pp. 18–39.
[47] Raviv, D., Bronstein, A. M., Bronstein, M. M., Kimmel, R. and Sochen, N., Affine-invariant geodesic geometry of deformable 3D shapes, Comput. Graphics, 35(3) (2011), pp. 692–697.
[48] Raviv, D., Bronstein, M. M., Bronstein, A. M. and Kimmel, R., Volumetric heat kernel signatures, 3DOR, 2010.
[49] Reuter, M., Wolter, F.-E. and Peinecke, N., Laplace-spectra as fingerprints for shape matching, In Proc. SPM, pages 101106, 2005.
[50] Rustamov, R. M., Laplace-Beltrami eigenfunctions for deformation invariant shape representation, In Proc. SGP, pages 225-233, 2007.
[51] Sivic, J. and Zisserman, A., Video google: a text retrieval approach to object matching in videos, In Proc. CVPR, 2003.
[52] Skraba, P., Ovsjanikov, M., Chazal, F. and Guibas, L., Persistence-based segmentation of de-formable shapes, In Proc. NORDIA, pages 4552, 2010.
[53] Sochen, N., On affine invariance in the beltrami framework for vision, In Proc. VLSM, 2001.
[54] Sun, J., Ovsjanikov, M. and Guibas, L., A Concise and provably informative multi-scale signature based on heat diffusion, Comput. Graphics Forum, 28(5) (2009), pp. 13831392.
[55] Sun, J., Ovsjanikov, M. and Guibas, L. J., A concise and provably informative multi-scale signature based on heat diffusion, In Proc. SGP, 2009.
[56] Thangudu, K., Practicality of Laplace Operator, Master’s thesis, The Ohio State University, Computer Science and Engineering Department, 2009.
[57] Thorstensen, N. and Keriven, R., Non-rigid shape matching using geometry and photometry, In Proc. CVPR, 2009.
[58] Toldo, R., Castellani, U. and Fusiello, A., Visual vocabulary signature for 3D object retrieval and partial matching, In Proc. 3DOR, 2009.
[59] Tomasi, C. and Manduchi, R., Bilateral fitering for gray and color images, In Proc. ICCV, pages 839846, 1998.
[60] Vranic, D.V., Saupe, D. and Richter, J., Tools for 3D-object retrieval: Karhunen-Loeve transform and spherical harmonics, In MMSP, pages 293-298, 2001.
[61] Wardetzky, M., Mathur, S., Kälberer, F. and Grinspun, E., Discrete Laplace operators: no free lunch, In Proc. SGP, 2008.
[62] Wu, C., Clipp, B., Li, X., Frahm, J.-M. and Pollefeys, M., 3D model matching with viewpoint-invariant patches (VIP), In Proc. CVPR, 2008.
[63] Wyngaerd, J.V., Combining texture and shape for automatic crude patch registration, DIM, pages 179186, 2003.
[64] Xu, G., Convergence of discrete Laplace-Beltrami operators over surfaces, Technical report, Institute of Computational Mathematics and Scientific/Engineering Computing, China, 2004.
[65] Yoon, K.-J., Prados, E. and Sturm, P., Joint estimation of shape and reflectance using multiple images with known illumination conditions, IJCV, 86(2-3) (2010), pp. 192210.
[66] Zaharescu, A., Boyer, E. and Horaud, R. P., Transformesh: a topology-adaptive mesh-based approach to surface evolution, In Proc. ACCV, 2007.
[67] Zaharescu, A., Boyer, E., Varanasi, K. and Horaud, R, Surface feature detection and description with applications to mesh matching, In Proc. CVPR, 2009.


Geometric and Photometric Data Fusion in Non-Rigid Shape Analysis

  • Artiom Kovnatsky (a1), Dan Raviv (a2), Michael M. Bronstein (a1), Alexander M. Bronstein (a3) and Ron Kimmel (a2)...


Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed