Papers

Using Very Deep Autoencoders for Content-Based Image Retrieval

• intro: ESANN 2011. Alex Krizhevsky, and Geoffrey E. Hinton
• paper: https://www.cs.toronto.edu/~hinton/absps/esann-deep-final.pdf
• paper: http://www.cs.toronto.edu/~fritz/absps/esann-deep-final.pdf

Learning High-level Image Representation for Image Retrieval via Multi-Task DNN using Clickthrough Data

• arxiv: http://arxiv.org/abs/1312.4740
• paper: http://legacy.openreview.net/document/90fc8dad-ad02-4ddc-ab06-e7b55706869d#90fc8dad-ad02-4ddc-ab06-e7b55706869d

Neural Codes for Image Retrieval

• intro: ECCV 2014
• project page: http://sites.skoltech.ru/compvision/projects/neuralcodes/
• arxiv: http://arxiv.org/abs/1404.1777
• github: https://github.com/arbabenko/Spoc

Efficient On-the-fly Category Retrieval using ConvNets and GPUs

• arxiv: http://arxiv.org/abs/1407.4764

Learning visual similarity for product design with convolutional neural networks

• intro: SIGGRAPH 2015
• paper: http://www.cs.cornell.edu/~kb/publications/SIG15ProductNet.pdf
• paper: http://dl.acm.org.sci-hub.cc/citation.cfm?doid=2809654.2766959

Exploiting Local Features from Deep Networks for Image Retrieval

• intro: CVPR DeepVision Workshop 2015
• arxiv: https://arxiv.org/abs/1504.05133

Visual Search at Pinterest

• intro: in Proceedings of the 21th ACM SIGKDD International Conference on Knowledge and Discovery and Data Mining, 2015
• arxiv: http://arxiv.org/abs/1505.07647
• blog: https://engineering.pinterest.com/blog/introducing-new-way-visually-search-pinterest

Cross-domain Image Retrieval with a Dual Attribute-aware Ranking Network

• intro: ICCV 2015
• intro: DARN, cross-entropy loss, triplet loss
• arxiv: http://arxiv.org/abs/1505.07922

Aggregating Deep Convolutional Features for Image Retrieval

• intro: ICCV 2015
• intro: Sum pooing
• arxiv: http://arxiv.org/abs/1510.07493

Particular object retrieval with integral max-pooling of CNN activations

• intro: use max-pooling to aggregate the deep descriptors, R-MAC (regional maximum activation of convolutions)
• arxiv: https://arxiv.org/abs/1511.05879

Group Invariant Deep Representations for Image Instance Retrieval

• arxiv: http://arxiv.org/abs/1601.02093

Where to Buy It: Matching Street Clothing Photos in Online Shops

• intro: ICCV 2015
• hmepage: http://www.tamaraberg.com/street2shop/
• paper: http://www.tamaraberg.com/papers/street2shop.pdf
• paper: http://www.cv-foundation.org/openaccess/content_iccv_2015/html/Kiapour_Where_to_Buy_ICCV_2015_paper.html

Natural Language Object Retrieval

• intro: CVPR 2015
• homepage: http://ronghanghu.com/text_obj_retrieval/
• arxiv: http://arxiv.org/abs/1511.04164
• slides: http://ronghanghu.com/slides/cvpr16_text_obj_retrieval_slides.pdf
• github: https://github.com/ronghanghu/natural-language-object-retrieval
• github: https://github.com/andrewliao11/Natural-Language-Object-Retrieval-tensorflow

Deep Image Retrieval: Learning global representations for image search

• intro: ECCV 2016
• project page: http://www.xrce.xerox.com/Research-Development/Computer-Vision/Learning-Visual-Representations/Deep-Image-Retrieval
• arxiv: https://arxiv.org/abs/1604.01325
• slides: http://www.slideshare.net/xavigiro/deep-image-retrieval-learning-global-representations-for-image-search

Bags of Local Convolutional Features for Scalable Instance Search

• intro: ICMR 2016. Best Poster Award at ICMR 2016.
• project page: https://imatge-upc.github.io/retrieval-2016-icmr/
• arxiv: https://arxiv.org/abs/1604.04653
• github: https://github.com/imatge-upc/retrieval-2016-icmr
• slides: http://www.slideshare.net/xavigiro/convolutional-features-for-instance-search

Faster R-CNN Features for Instance Search

• intro: DeepVision Workshop in CVPR 2016
• homepage: http://imatge-upc.github.io/retrieval-2016-deepvision/
• arxiv: http://arxiv.org/abs/1604.08893
• github: https://github.com/imatge-upc/retrieval-2016-deepvision

Where to Focus: Query Adaptive Matching for Instance Retrieval Using Convolutional Feature Maps

• intro: query adaptive matching (QAM), Feature Map Pooling, Overlapped Spatial Pyramid Pooling (OSPP)
• arxiv: https://arxiv.org/abs/1606.06811

Adversarial Training For Sketch Retrieval

• arxiv: http://arxiv.org/abs/1607.02748

Learning Compact Binary Descriptors with Unsupervised Deep Neural Networks

• intro: CVPR 2016. DeepBit
• paper: http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Lin_Learning_Compact_Binary_CVPR_2016_paper.pdf
• github: https://github.com/kevinlin311tw/cvpr16-deepbit

Fast Training of Triplet-based Deep Binary Embedding Networks

• intro: CVPR 2016
• arxiv: https://arxiv.org/abs/1603.02844
• paper: http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Zhuang_Fast_Training_of_CVPR_2016_paper.pdf
• bitbucket(official): https://bitbucket.org/jingruixiaozhuang/fast-training-of-triplet-based-deep-binary-embedding-networks

Deep Relative Distance Learning: Tell the Difference Between Similar Vehicles

• intro: CVPR 2016
• intro: vehicle re-identification, vehicle retrieval. coupled clusters loss
• paper: http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Liu_Deep_Relative_Distance_CVPR_2016_paper.pdf

DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations

• intro: CVPR 2016. FashionNet
• project page: http://personal.ie.cuhk.edu.hk/~lz013/projects/DeepFashion.html
• paper: http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Liu_DeepFashion_Powering_Robust_CVPR_2016_paper.pdf

CNN Image Retrieval Learns from BoW: Unsupervised Fine-Tuning with Hard Examples

• intro: ECCV 2016
• project page(paper+code+data): http://cmp.felk.cvut.cz/~radenfil/projects/siamac.html
• arxiv: https://arxiv.org/abs/1604.02426
• paper: http://cmp.felk.cvut.cz/~radenfil/publications/Radenovic-ECCV16.pdf
• code(Matlab): http://ptak.felk.cvut.cz/personal/radenfil/siamac/siaMAC_code.tar.gz

PicHunt: Social Media Image Retrieval for Improved Law Enforcement

• arxiv: http://arxiv.org/abs/1608.00905

SIFT Meets CNN: A Decade Survey of Instance Retrieval

• arxiv: http://arxiv.org/abs/1608.01807

The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies

• project page: http://sketchy.eye.gatech.edu/
• paper: http://www.cc.gatech.edu/~hays/tmp/sketchy-database.pdf
• github: https://github.com/janesjanes/sketchy

End-to-end Learning of Deep Visual Representations for Image Retrieval

• intro: ECCV 2016
• project page: http://www.xrce.xerox.com/Research-Development/Computer-Vision/Learning-Visual-Representations/Deep-Image-Retrieval
• arxiv: https://arxiv.org/abs/1610.07940

What Is the Best Practice for CNNs Applied to Visual Instance Retrieval?

• arxiv: https://arxiv.org/abs/1611.01640

Image Retrieval with Deep Local Features and Attention-based Keypoints

• arxiv: https://arxiv.org/abs/1612.05478

Internet-Based Image Retrieval Using End-to-End Trained Deep Distributions

• arxiv: https://arxiv.org/abs/1612.07697

Compression of Deep Neural Networks for Image Instance Retrieval

• intro: DCC 2017
• arxiv: https://arxiv.org/abs/1701.04923

Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval

• arxiv: https://arxiv.org/abs/1701.05003

Siamese Network of Deep Fisher-Vector Descriptors for Image Retrieval

• arxiv: https://arxiv.org/abs/1702.00338

Deep Geometric Retrieval

• arxiv: https://arxiv.org/abs/1702.06383

Context Aware Query Image Representation for Particular Object Retrieval

https://www.arxiv.org/abs/1703.01226

SVDNet for Pedestrian Retrieval

• intro: On the Market-1501 dataset, rank-1 accuracy is improved from 55.2% to 80.5% for CaffeNet, and from 73.8% to 83.1% for ResNet-50
• arxiv: https://arxiv.org/abs/1703.05693

An End-to-End Approach to Natural Language Object Retrieval via Context-Aware Deep Reinforcement Learning

https://arxiv.org/abs/1703.07579

AMC: Attention guided Multi-modal Correlation Learning for Image Search

• intro: CVPR 2017
• arxiv: https://arxiv.org/abs/1704.00763
• github: https://github.com/kanchen-usc/amc_att

Deep Learning for Content-Based, Cross-Modal Retrieval of Videos and Music

• arxiv: https://arxiv.org/abs/1704.06761
• supplementary: https://youtu.be/ZyINqDMo3Fg

Deep image representations using caption generators

• intro: ICME 2017
• arxiv: https://arxiv.org/abs/1705.09142

Visual Search at eBay

• intro: 23rd SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2017
• arxiv: https://arxiv.org/abs/1706.03154

Sampling Matters in Deep Embedding Learning

• intro: UT Austin & A9/Amazon
• keywords: distance weighted sampling
• arxiv: https://arxiv.org/abs/1706.07567

One-Shot Fine-Grained Instance Retrieval

• intro: ACM MM 2017
• arxiv: https://arxiv.org/abs/1707.00811

Selective Deep Convolutional Features for Image Retrieval

• intro: ACM MM 2017
• arxiv: https://arxiv.org/abs/1707.00809

Class-Weighted Convolutional Features for Visual Instance Search

• intro: BMVC 2017. Universitat Politecnica de Catalunya Barcelona & CSIRO
• project page: http://imatge-upc.github.io/retrieval-2017-cam/
• arxiv: https://arxiv.org/abs/1707.02581
• github: https://github.com/imatge-upc/retrieval-2017-cam

Deep Binaries: Encoding Semantic-Rich Cues for Efficient Textual-Visual Cross Retrieval

• intro: ICCV 2017
• arxiv: https://arxiv.org/abs/1708.02531

Learning a Repression Network for Precise Vehicle Search

https://arxiv.org/abs/1708.02386

SUBIC: A supervised, structured binary code for image search

• intro: ICCV 2017 (Spotlight). Technicolor & INRIA Rennes & Amazon
• arxiv: https://arxiv.org/abs/1708.02932

Hashing

Supervised Hashing for Image Retrieval via Image Representation Learning

• intro: AAAI 2014. Sun Yat-Sen University & National University of Singapore
• keywords: CNNH (Convolutional Neural Network Hashing)
• paper: www.aaai.org/ocs/index.php/AAAI/AAAI14/paper/download/8137/8861
• slides: https://pdfs.semanticscholar.org/f633/8f23860f9c4808586bbc7e8907d33836147f.pdf

Simultaneous Feature Learning and Hash Coding with Deep Neural Networks

• intro: CVPR 2015. Sun Yat-Sen University & National University of Singapore
• keywords: NINH (NIN Hashing), DNNH (Deep Neural Network Hashing)
• arxiv: https://arxiv.org/abs/1504.03410
• paper: http://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Lai_Simultaneous_Feature_Learning_2015_CVPR_paper.pdf

Hashing by Deep Learning

• intro: IBM T. J. Watson Research Center
• paper: http://www.ee.columbia.edu/~wliu/WeiLiu_DLHash.pdf

Deep Semantic Ranking Based Hashing for Multi-Label Image Retrieval

• intro: CVPR 2015. DSRH (Deep Semantic Ranking Hashing)
• arxiv: http://arxiv.org/abs/1501.06272

Deep Learning of Binary Hash Codes for Fast Image Retrieval

• intro: CVPR Workshop 2015
• keywords: MNIST, CIFAR-10, Yahoo-1M. DLBHC (Deep Learning of Binary Hash Codes)
• paper: http://www.iis.sinica.edu.tw/~kevinlin311.tw/cvprw15.pdf
• github: https://github.com/kevinlin311tw/caffe-cvprw15

Supervised Learning of Semantics-Preserving Hashing via Deep Neural Networks for Large-Scale Image Search

• intro: SSDH
• arxiv: http://arxiv.org/abs/1507.00101
• github: https://github.com/kevinlin311tw/Caffe-DeepBinaryCode

Bit-Scalable Deep Hashing with Regularized Similarity Learning for Image Retrieval and Person Re-identification

• intro: IEEE Transactions on Image Processing 2015
• keywords: DRSCH (Deep Regularized Similarity Comparison Hashing)
• project page: http://vision.sysu.edu.cn/projects/deephashing/
• arxiv: https://arxiv.org/abs/1508.04535
• github: https://github.com/ruixuejianfei/BitScalableDeepHash

Deep Supervised Hashing for Fast Image Retrieval

• intro: CVPR 2016
• keywords: DSH (Deep Supervised Hashing)
• paper: http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Liu_Deep_Supervised_Hashing_CVPR_2016_paper.pdf
• paper: http://www.jdl.ac.cn/doc/2011/201711214443668218_deep%20supervised%20hashing%20for%20fast%20image%20retrieval_cvpr2016.pdf
• github: https://github.com/lhmRyan/deep-supervised-hashing-DSH

Deep Hashing Network for Efficient Similarity Retrieval

• intro: AAAI 2016
• paper: http://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/view/12039

Feature Learning based Deep Supervised Hashing with Pairwise Labels

• intro: IJCAI 2016
• arxiv: https://arxiv.org/abs/1511.03855
• paper: https://www.ijcai.org/Proceedings/16/Papers/245.pdf
• paper: https://cs.nju.edu.cn/lwj/paper/IJCAI16_DPSH.pdf
• code: http://cs.nju.edu.cn/lwj/code/DPSH_code.rar

Deep Cross-Modal Hashing

https://arxiv.org/abs/1602.02255

SSDH: Semi-supervised Deep Hashing for Large Scale Image Retrieval

• arxiv: http://arxiv.org/abs/1607.08477

Deep Semantic-Preserving and Ranking-Based Hashing for Image Retrieval

• intro: Microsoft
• paper: http://www.microsoft.com/en-us/research/wp-content/uploads/2016/08/Deep-Semantic-Preserving-and-Ranking-Based-Hashing-for-Image-Retrieval.pdf

Deep Hashing: A Joint Approach for Image Signature Learning

• arxiv: http://arxiv.org/abs/1608.03658

Transitive Hashing Network for Heterogeneous Multimedia Retrieval

• intro: state of the art on NUS-WIDE, ImageNet-YahooQA
• arxiv: http://arxiv.org/abs/1608.04307

Deep Residual Hashing

• arxiv: https://arxiv.org/abs/1612.05400

Deep Region Hashing for Efficient Large-scale Instance Search from Images

• intro: Columbia University & University of Electronic Science and Technology of China
• arxiv: https://arxiv.org/abs/1701.07901

HashNet: Deep Learning to Hash by Continuation

• intro: ICCV 2017. Tsinghua University
• arxiv: https://arxiv.org/abs/1702.00758
• github: https://github.com/thuml/HashNet

Unsupervised Triplet Hashing for Fast Image Retrieval

• arxiv: https://www.arxiv.org/abs/1702.08798

Deep Sketch Hashing: Fast Free-hand Sketch-Based Image Retrieval

• intro: CVPR 2017 spotlight paper
• arxiv: https://arxiv.org/abs/1703.05605

Learning Robust Hash Codes for Multiple Instance Image Retrieval

• arxiv: https://arxiv.org/abs/1703.05724

Simultaneous Feature Aggregating and Hashing for Large-scale Image Search

• intro: CVPR 2017
• arxiv: https://arxiv.org/abs/1704.00860

Learning to Hash

• blog: https://cs.nju.edu.cn/lwj/L2H.html

Hashing as Tie-Aware Learning to Rank

https://arxiv.org/abs/1705.08562

Deep Hashing Network for Unsupervised Domain Adaptation

• intro: CVPR 2017
• arxiv: https://arxiv.org/abs/1706.07522
• github(MatConvNet): https://github.com/hemanthdv/da-hash

Pruning Convolutional Neural Networks for Image Instance Retrieval

https://arxiv.org/abs/1707.05455

Projects

HABIR哈希图像检索工具箱

• intro: Various hashing methods for image retrieval and serves as the baselines
• blog: http://yongyuan.name/habir/
• github: https://github.com/willard-yuan/hashing-baseline-for-image-retrieval

Video Indexing / Retrieval

Face Video Retrieval via Deep Learning of Binary Hash Representations

• paper: https://www.aaai.org/ocs/index.php/AAAI/AAAI16/paper/download/11893/12117

Deep Learning Based Semantic Video Indexing and Retrieval

• arxiv: https://arxiv.org/abs/1601.07754

Learning Joint Representations of Videos and Sentences with Web Image Search

• intro: 4th Workshop on Web-scale Vision and Social Media (VSM), ECCV 2016
• arxiv: http://arxiv.org/abs/1608.02367

Multi-View Product Image Search Using ConvNets Features

• arxiv: http://arxiv.org/abs/1608.03462

Generalisation and Sharing in Triplet Convnets for Sketch based Visual Search

• arxiv: https://arxiv.org/abs/1611.05301

Binary Subspace Coding for Query-by-Image Video Retrieval

• arxiv: https://arxiv.org/abs/1612.01657

Action Search: Learning to Search for Human Activities in Untrimmed Videos

https://arxiv.org/abs/1706.04269

Deep Supervised Hashing with Triplet Labels

• intro: ACCV 2016
• arxiv: https://arxiv.org/abs/1612.03900

Supervised Deep Hashing for Hierarchical Labeled Data

• arxiv: https://arxiv.org/abs/1704.02088

Localizing Moments in Video with Natural Language

• intro: ICCV 2017
• arxiv: https://arxiv.org/abs/1708.01641

Learning to Rank

Simple to Complex Cross-modal Learning to Rank

• intro: Xi’an Jiaotong University & University of Technology Sydney & National University of Singapore & CMU
• arxiv: https://arxiv.org/abs/1702.01229

Deep Metric Learning

Deep metric learning using Triplet network

• arxiv: https://arxiv.org/abs/1412.6622
• slides: http://tce.technion.ac.il/wp-content/uploads/sites/8/2016/01/Elad-Hofer.pdf
• github: https://github.com/eladhoffer/TripletNet

Learnable Structured Clustering Framework for Deep Metric Learning

• arxiv: https://arxiv.org/abs/1612.01213

Deep Metric Learning via Lifted Structured Feature Embedding

• intro: CVPR 2016
• project page(code+data): http://cvgl.stanford.edu/projects/lifted_struct/
• paper: http://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Song_Deep_Metric_Learning_CVPR_2016_paper.pdf
• paper: http://cvgl.stanford.edu/papers/song_cvpr16.pdf
• github: https://github.com/rksltnl/Deep-Metric-Learning-CVPR16
• dataset: ftp://cs.stanford.edu/cs/cvgl/Stanford_Online_Products.zip

Cross-modal Deep Metric Learning with Multi-task Regularization

• intro: ICME 2017
• arxiv: https://arxiv.org/abs/1703.07026

Smart Mining for Deep Metric Learning

https://arxiv.org/abs/1704.01285

DarkRank: Accelerating Deep Metric Learning via Cross Sample Similarities Transfer

• intro: TuSimple
• keywords: pedestrian re-identification
• arxiv: https://arxiv.org/abs/1707.01220

Deep Metric Learning with Angular Loss

• intro: ICCV 2017
• arxiv: https://arxiv.org/abs/1708.01682

Talks / Slides

TiefVision: end-to-end image similarity search engine

• intro: It covers image classification, image location ( OverFeat ) and image similarity ( Deep Ranking).
• slides: https://docs.google.com/presentation/d/16hrXJhOzkbmla9AL7JCreCuBsa5L80gm71Pfrjo7F9Y/edit#slide=id.p
• github: https://github.com/paucarre/tiefvision

Projects

图像检索：CNN卷积神经网络与实战

CNN for Image Retrieval

• blog: http://yongyuan.name/blog/CBIR-CNN-and-practice.html
• github: https://github.com/willard-yuan/CNN-for-Image-Retrieval
• demo: http://yongyuan.name/pic/

Visual Search Server

• intro: A simple implementation of Visual Search using features extracted from Tensorflow inception model and Approximate Nearest Neighbors
• github: https://github.com/AKSHAYUBHAT/VisualSearchServer

Vehicle Retrieval: vehicle image retrieval using k CNNs ensemble method

• intro: ranked 1st and won the special prize in the final of the 3rd National Gradute Contest on Smart-CIty Technology and Creative Design, China
• project page: https://www.pkuml.org/resources/pku-vehicleid.html
• github: https://github.com/iamhankai/vehicle-retrieval-kCNNs

A visual search engine based on Elasticsearch and Tensorflow

• keywords: faster r-cnn
• github: https://github.com/tuan3w/visual_search

Blogs

Where can I buy a chair like that? – This app will tell you

• blog: http://www.news.cornell.edu/stories/2016/08/where-can-i-buy-chair-app-will-tell-you

Using Sketches to Search for Products Online

• homepage: http://sketchx.eecs.qmul.ac.uk/
• blog: https://news.developer.nvidia.com/using-sketches-to-search-for-products-online/

Tutorials

Deep Image Retrieval: Learning global representations for image search

• youtube: https://www.youtube.com/watch?v=yT52xDML6ys

Image Instance Retrieval: Overview of state-of-the-art

• youtube: https://www.youtube.com/watch?v=EYq-rpaZn1o