Skip to content

cossio/RestrictedBoltzmannMachines.jl

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Jan 25, 2025
7a71503 · Jan 25, 2025
Apr 8, 2024
Jan 25, 2025
Sep 15, 2023
Aug 5, 2023
Jan 25, 2025
Jan 25, 2025
Jan 25, 2025
Jan 25, 2025
Jan 25, 2025
Jan 25, 2025
Jan 25, 2025
Apr 5, 2023
Nov 12, 2021
Jan 25, 2025
Jan 25, 2025
Jan 25, 2025
Jan 12, 2022

Repository files navigation

RestrictedBoltzmannMachines Julia package

Train and sample Restricted Boltzmann machines in Julia.

Installation

This package is registered. Install with:

import Pkg
Pkg.add("RestrictedBoltzmannMachines")

This package does not export any symbols. Since the name RestrictedBoltzmannMachines is long, it can be imported as:

import RestrictedBoltzmannMachines as RBMs

Usage with CUDA

We define two functions, cpu and gpu (similar to Flux.jl), to move RBM to/from the CPU and GPU.

import CUDA # if you want to use the GPU, need to import this
using RestrictedBoltzmannMachines: BinaryRBM, cpu, gpu

rbm = BinaryRBM(randn(5), randn(3), randn(5,3)) # in CPU

# copy to GPU
rbm_cu = gpu(rbm)

# ... do some things with rbm_cu on the GPU (e.g. training, sampling)

# copy back to CPU
rbm = cpu(rbm_cuda)

CenteredRBM

Train and sample centered Restricted Boltzmann machines in Julia. See [Melchior et al] for the definition of centered. Consider an RBM with binary units. Then the centered variant has energy defined by:

E ( v , h ) = i a i v i μ b μ h μ i μ w i μ ( v i c i ) ( h μ d μ )

with offset parameters c i , d μ . Typically c i , d μ are set to approximate the average activities of v i and h μ , respectively, as this seems to help training (see [Montavon et al]).

StandardizedRBM

Train and sample a standardized Restricted Boltzmann machine in Julia. The energy is given by:

E ( v , h ) = i θ i v i μ θ μ h μ i μ w i μ v i λ i σ i h μ λ μ σ μ

with some offset parameters λ i , λ μ and scaling parameters σ i , σ μ . Usually λ i , λ μ track the mean activities of visible and hidden units, while σ i , σ μ track their standard deviations.

Related packages

Adversarially constrained RBMs:

Stacked tempering:

References

  • Montavon, Grégoire, and Klaus-Robert Müller. "Deep Boltzmann machines and the centering trick." Neural networks: tricks of the trade. Springer, Berlin, Heidelberg, 2012. 621-637.
  • Melchior, Jan, Asja Fischer, and Laurenz Wiskott. "How to center deep Boltzmann machines." The Journal of Machine Learning Research 17.1 (2016): 3387-3447.

Citation

If you use this package in a publication, please cite:

  • Jorge Fernandez-de-Cossio-Diaz, Simona Cocco, and Remi Monasson. "Disentangling representations in Restricted Boltzmann Machines without adversaries." Physical Review X 13, 021003 (2023).

Or you can use the included CITATION.bib.

About

Train and sample Restricted Boltzmann machines in Julia

Topics

Resources

License

Citation

Stars

Watchers

Forks

Packages

No packages published