Restricted Boltzmann Machine (RBM)

Sparse Coding

Auto-encoder

Papers

On Random Weights and Unsupervised Feature Learning

• intro: ICML 2011
• paper: http://www.robotics.stanford.edu/~ang/papers/icml11-RandomWeights.pdf

Unsupervised Learning of Spatiotemporally Coherent Metrics

• paper: http://arxiv.org/abs/1412.6056
• code: https://github.com/jhjin/flattened-cnn

Unsupervised Learning of Visual Representations using Videos

• intro: ICCV 2015
• project page: http://www.cs.cmu.edu/~xiaolonw/unsupervise.html
• arxiv: http://arxiv.org/abs/1505.00687
• paper: http://www.cs.toronto.edu/~nitish/depth_oral.pdf
• github: https://github.com/xiaolonw/caffe-video_triplet

Unsupervised Visual Representation Learning by Context Prediction

• intro: ICCV 2015
• homepage: http://graphics.cs.cmu.edu/projects/deepContext/
• arxiv: http://arxiv.org/abs/1505.05192
• github: https://github.com/cdoersch/deepcontext

Unsupervised Learning on Neural Network Outputs

• intro: “use CNN trained on the ImageNet of 1000 classes to the ImageNet of over 20000 classes”
• arxiv: http://arxiv.org/abs/1506.00990
• github: https://github.com/yaolubrain/ULNNO

Unsupervised Domain Adaptation by Backpropagation

• intro: ICML 2015
• project page: http://sites.skoltech.ru/compvision/projects/grl/
• paper: http://sites.skoltech.ru/compvision/projects/grl/files/paper.pdf
• github: https://github.com/ddtm/caffe/tree/grl

Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles

• arxiv: http://arxiv.org/abs/1603.09246
• notes: http://www.inference.vc/notes-on-unsupervised-learning-of-visual-representations-by-solving-jigsaw-puzzles/

Tagger: Deep Unsupervised Perceptual Grouping

• intro: NIPS 2016
• arxiv: https://arxiv.org/abs/1606.06724
• github: https://github.com/CuriousAI/tagger

Regularization for Unsupervised Deep Neural Nets

• arxiv: http://arxiv.org/abs/1608.04426

Sparse coding: A simple exploration

• blog: https://blog.metaflow.fr/sparse-coding-a-simple-exploration-152a3c900a7c#.o7g2jk9zi
• github: https://github.com/metaflow-ai/blog/tree/master/sparse-coding

Navigating the unsupervised learning landscape

• blog: https://culurciello.github.io//tech/2016/06/10/unsup.html

Unsupervised Learning using Adversarial Networks

• intro: Facebook AI Research
• youtube: https://www.youtube.com/watch?v=lalg1CuNB30

Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction

• intro: UC Berkeley
• project page: https://richzhang.github.io/splitbrainauto/
• arxiv: https://arxiv.org/abs/1611.09842
• github: https://github.com/richzhang/splitbrainauto

Learning Features by Watching Objects Move

• intro: CVPR 2017. Facebook AI Research & UC Berkeley
• arxiv: https://arxiv.org/abs/1612.06370
• github((Caffe+Torch): https://github.com/pathak22/unsupervised-video

CNN features are also great at unsupervised classification

• intro: Arts et Métiers ParisTech
• arxiv: https://arxiv.org/abs/1707.01700

Clustering

Deep clustering: Discriminative embeddings for segmentation and separation

• arxiv: https://arxiv.org/abs/1508.04306
• github(Keras): https://github.com/jcsilva/deep-clustering

Unsupervised Deep Embedding for Clustering Analysis

• intro: ICML 2016. Deep Embedded Clustering (DEC)
• arxiv: https://arxiv.org/abs/1511.06335
• github: https://github.com/piiswrong/dec

Joint Unsupervised Learning of Deep Representations and Image Clusters

• intro: CVPR 2016
• arxiv: https://arxiv.org/abs/1604.03628
• github(Torch): https://github.com/jwyang/joint-unsupervised-learning

Single-Channel Multi-Speaker Separation using Deep Clustering

• arxiv: http://arxiv.org/abs/1607.02173

Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering

• arxiv: https://arxiv.org/abs/1610.04794

Deep Unsupervised Clustering with Gaussian Mixture Variational

• arxiv: https://arxiv.org/abs/1611.02648
• github: https://github.com/Nat-D/GMVAE

Variational Deep Embedding: A Generative Approach to Clustering

• arxiv: https://arxiv.org/abs/1611.05148

A new look at clustering through the lens of deep convolutional neural networks

• intro: University of Central Florida & Purdue University
• arxiv: https://arxiv.org/abs/1706.05048

Auto-encoder

Auto-Encoding Variational Bayes

• arxiv: http://arxiv.org/abs/1312.6114

The Potential Energy of an Autoencoder

• intro: PAMI 2014
• paper: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.698.4921&rep=rep1&type=pdf

Importance Weighted Autoencoders

• paper: http://arxiv.org/abs/1509.00519
• github: https://github.com/yburda/iwae

Review of Auto-Encoders

• intro: Piotr Mirowski, Microsoft Bing London, 2014
• slides: https://piotrmirowski.files.wordpress.com/2014/03/piotrmirowski_2014_reviewautoencoders.pdf
• github: https://github.com/piotrmirowski/Tutorial_AutoEncoders/

Stacked What-Where Auto-encoders

• arxiv: http://arxiv.org/abs/1506.02351

Ladder Variational Autoencoders

How to Train Deep Variational Autoencoders and Probabilistic Ladder Networks

• arxiv:http://arxiv.org/abs/1602.02282
• github: https://github.com/casperkaae/LVAE

Rank Ordered Autoencoders

• arxiv: http://arxiv.org/abs/1605.01749
• github: https://github.com/paulbertens/rank-ordered-autoencoder

Decoding Stacked Denoising Autoencoders

• arxiv: http://arxiv.org/abs/1605.02832

Keras autoencoders (convolutional/fcc)

• github: https://github.com/nanopony/keras-convautoencoder

Building Autoencoders in Keras

• blog: http://blog.keras.io/building-autoencoders-in-keras.html

Review of auto-encoders

• intro: Tutorial code for Auto-Encoders, implementing Marc’Aurelio Ranzato’s Sparse Encoding Symmetric Machine and testing it on the MNIST handwritten digits data.
• paper: https://github.com/piotrmirowski/Tutorial_AutoEncoders/blob/master/PiotrMirowski_2014_ReviewAutoEncoders.pdf
• github: https://github.com/piotrmirowski/Tutorial_AutoEncoders

Autoencoders: Torch implementations of various types of autoencoders

• intro: AE / SparseAE / DeepAE / ConvAE / UpconvAE / DenoisingAE / VAE / AdvAE
• github: https://github.com/Kaixhin/Autoencoders

Tutorial on Variational Autoencoders

• arxiv: http://arxiv.org/abs/1606.05908
• github: https://github.com/cdoersch/vae_tutorial

Variational Autoencoders Explained

• blog: http://kvfrans.com/variational-autoencoders-explained/
• github: https://github.com/kvfrans/variational-autoencoder

Introducing Variational Autoencoders (in Prose and Code)

• blog: http://blog.fastforwardlabs.com/post/148842796218/introducing-variational-autoencoders-in-prose-and

Under the Hood of the Variational Autoencoder (in Prose and Code)

• blog: http://blog.fastforwardlabs.com/post/149329060653/under-the-hood-of-the-variational-autoencoder-in

The Unreasonable Confusion of Variational Autoencoders

• blog: https://jaan.io/unreasonable-confusion/

Variational Autoencoder for Deep Learning of Images, Labels and Captions

• intro: NIPS 2016. Duke University & Nokia Bell Labs
• paper: http://people.ee.duke.edu/~lcarin/Yunchen_nips_2016.pdf

Convolutional variational autoencoder with PyMC3 and Keras

http://nbviewer.jupyter.org/github/taku-y/pymc3/blob/89b8634a2fd30ef96429953558bf360132b6153f/docs/source/notebooks/convolutional_vae_keras_advi.ipynb

Pixelvae: A Latent Variable Model For Natural Images

• paper: http://openreview.net/pdf?id=BJKYvt5lg

beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework

• paper: http://openreview.net/pdf?id=Sy2fzU9gl
• github: https://github.com/crcrpar/chainer-VAE

Variational Lossy Autoencoder

• arxiv: https://arxiv.org/abs/1611.02731

Convolutional Autoencoders

• blog: https://pgaleone.eu/neural-networks/2016/11/24/convolutional-autoencoders/

Convolutional Autoencoders in Tensorflow

• blog: https://pgaleone.eu/neural-networks/deep-learning/2016/12/13/convolutional-autoencoders-in-tensorflow/

A Deep Convolutional Auto-Encoder with Pooling - Unpooling Layers in Caffe

• arxiv: https://arxiv.org/abs/1701.04949

RBM (Restricted Boltzmann Machine)

Papers

Deep Boltzmann Machines

• author: Ruslan Salakhutdinov, Geoffrey Hinton
• paper: http://www.cs.toronto.edu/~hinton/absps/dbm.pdf

On the Equivalence of Restricted Boltzmann Machines and Tensor Network States

• arxiv: https://arxiv.org/abs/1701.04831
• github: https://github.com/yzcj105/rbm2mps

Blogs

A Tutorial on Restricted Boltzmann Machines

http://xiangjiang.live/2016/02/12/a-tutorial-on-restricted-boltzmann-machines/

Dreaming of names with RBMs

• blog: http://colinmorris.github.io/blog/dreaming-rbms
• github: https://github.com/colinmorris/char-rbm

on Cheap Learning: Partition Functions and RBMs

• blog: https://charlesmartin14.wordpress.com/2016/09/10/on-cheap-learning-partition-functions-and-rbms/

Improving RBMs with physical chemistry

• blog: https://charlesmartin14.wordpress.com/2016/10/21/improving-rbms-with-physical-chemistry/
• github: https://github.com/charlesmartin14/emf-rbm/blob/master/EMF_RBM_Test.ipynb

Projects

Restricted Boltzmann Machine (Haskell)

• intro: “This is an implementation of two machine learning algorithms, Contrastive Divergence and Back-propagation.”
• github: https://github.com/aeyakovenko/rbm

tensorflow-rbm: Tensorflow implementation of Restricted Boltzman Machine

• intro: Tensorflow implementation of Restricted Boltzman Machine for layerwise pretraining of deep autoencoders.
• github: https://github.com/meownoid/tensorfow-rbm

Videos

Modelling a text corpus using Deep Boltzmann Machines

• youtube: https://www.youtube.com/watch?v=uju4RXEniA8

Foundations of Unsupervised Deep Learning

• intro: Ruslan Salakhutdinov [CMU]
• youtube: https://www.youtube.com/watch?v=rK6bchqeaN8
• mirror: https://pan.baidu.com/s/1mi4nCow
• sildes: http://www.cs.cmu.edu/~rsalakhu/talk_unsup.pdf