Learning a hybrid similarity measure for image retrieval

Jun Wu, Hong Shen, Yi Dong Li, Zhi Bo Xiao, Ming Yu Lu, Chun Li Wang

Research output: Contribution to journalArticlepeer-review

53 Citations (Scopus)


Learning similarity measure from relevance feedback has become a promising way to enhance the image retrieval performance. Existing approaches mainly focus on taking short-term learning experience to identify a visual similarity measure within a single query session, or applying long-term learning methodology to infer a semantic similarity measure crossing multiple query sessions. However, there is still a big room to elevate the retrieval effectiveness, because little is known in taking the relationship between visual similarity and semantic similarity into account. In this paper, we propose a novel hybrid similarity learning scheme to preserve both visual and semantic resemblance by integrating short-term with long-term learning processes. Concretely, the proposed scheme first learns a semantic similarity from the users' query log, and then, taking this as prior knowledge, learns a visual similarity from a mixture of labeled and unlabeled images. In particular, unlabeled images are exploited for the relevant and irrelevant classes differently and the visual similarity is learned incrementally. Finally, a hybrid similarity measure is produced by fusing the visual and semantic similarities in a nonlinear way for image ranking. An empirical study shows that using hybrid similarity measure for image retrieval is beneficial, and the proposed algorithm achieves better performance than some existing approaches.

Original languageEnglish
Pages (from-to)2927-2939
Number of pages13
JournalPattern Recognition
Issue number11
Publication statusPublished - Nov 2013
Externally publishedYes


  • Hybrid similarity measure
  • Image retrieval
  • Long-term learning
  • Relevance feedback
  • Short-term learning


Dive into the research topics of 'Learning a hybrid similarity measure for image retrieval'. Together they form a unique fingerprint.

Cite this