(Click images to enlarge)

In this research area we study probabilistic systems that autonomously learn from data and are able to recognize a complex data point as a combination of its components. The project builds up on component extraction approaches such as principle component analysis (PCA), independent component analysis (ICA), sparse coding, and non-negative matrix factorization (NMF). All these approaches assume linear superposition of components, which is a valid assumption in only a limited range of cases (e.g., for sound waveforms). In vision, as well as in other modalities, components interact non-linearly. This project therefore focuses on probabilistic generative models that combine components non-linearly. Furthermore, we advance the prior assumptions of standard approaches and aim at inferring prior structure. Research includes (1) the derivation and investigation of algorithms for non-linear generative models, (2) the development and application of approximation schemes that allow to train such models, and (3) the development of hierarchical extensions of non-linear models and models with structured priors.

Python based source code of the 'Binary Sparse Coding (BSC)', 'Maximal Causes Analysis (MCA)'  and 'Spike-and-slab/Gaussian Sparse Coding (GSC)' algorithms are now available for download. The software packages implement MPI based parallelization and can be readily run on a single or multi-core/parallel architecture. Please see the corresponding 'Readme' files for more details about individual packages.

All the packages are available under the Academic Free License (AFL) v3.0.

GSC: [Gzip file, Readme]

J. Lücke^* and A.-S. Sheikh^* (2012).
A Closed-Form EM Algorithm for Sparse Coding and Its Application to Source Separation (arXiv version, bibtex).
International Conference on Latent Variable Analysis and Signal Separation (LVA/ICA), 213-221, 2012.
^*joint first authorship

BSC: [Gzip file, Readme]

M. Henniges, G. Puertas, J. Bornschein, J. Eggert, and J. Lücke (2010). 
Binary Sparse Coding (pdf, bibtex). 
Proc. LVA/ICA 2010, LNCS 6365, 450-457

MCA: [Gzip file, Readme]

G. Puertas^*, J. Bornschein^*, and J. Lücke (2010). 
The Maximal Causes of Natural Scenes are Edge Filters (pdf, bibtex, supplement). 
Advances in Neural Information Processing Systems 23, 1939-1947, 2010.
^*joint first authorship

• J.A. Shelton, A.-S. Sheikh, J. Bornschein, P. Sterne and J. Lücke (2015).
  Nonlinear Spike-and-slab Sparse Coding for Interpretable Image Encoding
  PLoS One 10(5): e0124088 (online access, bibtex)
   
• A.-S. Sheikh, J. A. Shelton, J. Lücke (2014). 
  A Truncated EM Approach for Spike-and-Slab Sparse Coding. 
  Journal of Machine Learning Research, 15:2653-2687. (online access, bibtex)
   
• M. Henniges, R. E. Turner, M. Sahani, J. Eggert, J. Lücke (2014). 
  Efficient Occlusive Components Analysis.
  Journal of Machine Learning Research, 15:2689-2722. (online access, bibtex)
   
• G. Exarchakis, M. Henniges, J. Eggert, and J. Lücke (2012).
  Ternary Sparse Coding (bibtex).
  International Conference on Latent Variable Analysis and Signal Separation (LVA/ICA), 204-212, 2012.


• J. Lücke and M. Henniges (2012).
  Closed-Form Entropy Limits – A Tool to Monitor Likelihood Optimization of Probabilistic Generative Models (pdf, bibtex).
  AI & Statistics (AISTATS 15), 731-740, 2012.


• J. A. Shelton, J. Bornschein, A.-S. Sheikh, P. Berkes, and J. Lücke (2011).
  Select and Sample — A Model of Efficient Neural Inference and Learning (pdf, bibtex).
  Advances in Neural Information Processing Systems 24, 2618-2626, 2011.


• J. Bornschein, M. Henniges, G. Puertas, and J. Lücke (2011).
  Sparse codes of V1 simple-cells and the emergence of globular receptive fields – a comparative study
  Proc. COSYNE. (abstract, poster)


• J. Lücke and J. Eggert (2010).
  Expectation Truncation and the Benefits of Preselection in Training Generative Models. (pdf, bibtex, animations, talk).
  Journal of Machine Learning Research 11:2855-2900, 2010.


• J. Bornschein and J. Lücke (2009).
  Applications of Non-linear Component Extraction to Spectrogram Representations Of Auditory Data
  Frontiers in Compuational Neuroscience, Proc. BCCN (online access).


• J. Lücke, R. Turner, M. Sahani, and M. Henniges (2009).
  Occlusive Components Analysis (pdf, bibtex, supplementary).
  Advances in Neural Information Processing Systems 22, 1069-1077.


• J. Lücke and M. Sahani (2008).
  Maximal Causes for Non-linear Component Extraction (pdf, bibtex).
  Journal of Machine Learning Research 9:1227-1267.


• J. Lücke and M. Sahani (2007).
  Generalized Softmax Networks for Non-Linear Component Extraction (bibtex).
  Proc. ICANN, Springer, LNCS 4668, 657-667.

The papers listed above have been published after peer review in different journals. These journals remain the only definitive repository of the content. Copyright and all rights therein are usually retained by the respective publishers. These materials may not be copied or reposted without their explicit permission. Use for scholarly purposes only.