==================================================
Gaussian-Bernoulli RBM in CUDA.
Written by Alex Krizhevsky (kriz@cs.toronto.edu).
Jan 4, 2010
==================================================

This is a RBM with Gaussian visible units and Bernoulli hidden units; it can be trained on any real-valued data.

This program is written as a python extension, so to compile it you will need a few extra libraries.

These are:

1. Python 2.5/2.6
2. Numpy
3. ATLAS
4. CUDA 2.3

If you use Ubuntu you can install all of these from the package manager without much hassle.

When you're done, you need to set the following environment variables (exact path may vary):
export PYTHON_INCLUDE=/usr/include/python2.6/
export NUMPY_INCLUDE=/usr/lib/python2.6/dist-packages/numpy/core/include/numpy/

In addition, you must recompile your CUDA SDK in such a way that it is possible to use it from a shared object (at least if you are on a 64-bit system).
To do this, change the following lines in <CUDA_SDK_PATH>/C/common/common.mk from this:

# Compilers
NVCC       := $(CUDA_INSTALL_PATH)/bin/nvcc
CXX        := g++
CC         := gcc
LINK       := g++ -fPIC

to this:

# Compilers
NVCC       := $(CUDA_INSTALL_PATH)/bin/nvcc -Xcompiler -fPIC
CXX        := g++ -fPIC
CC         := gcc -fPIC
LINK       := g++ -fPIC

and recompile it the SDK.

I have only tested this code on 64-bit systems with GTX 280-class GPUs.

The makefile will create a shared object somewhere under the ./bin subdirectory. Make a symbolic link to that object in the main directory. On my system, the command is:

ln -s ./bin/linux/release/_nvrbm.so _nvrbm.so

The RBM expects that the data you feed it is in single precision floating point, and the cases arranged in rows.

Run:
python nvrbm.py
to get a list of the options that the RBM takes. Many of the options have sensible default values.

Assuming you have data you want to train on, here is one way to run the RBM:

python nvrbm.py --data-path=/storage/tiny/tinyimages-batches_del1000/ --num-hid=8192 --sigma=69 --test-range=199 --train-range=1-198 --save-path=/storage/tmp

This will train an RBM with 8192 hidden units on a subset of the 80 million tiny images dataset. The subset happens to have roughly 1.5 million images. This subset has been normalized to have zero mean, but not unit variance. I have observed the average standard deviation of its components to be 69, which explains the "--sigma=69".
