Neural Foundations · 2016
Variational Lossy Autoencoder
Explained, through a bits-back coding argument, why a variational autoencoder paired with a powerful autoregressive decoder learns to ignore its latent code, and used that understanding to control which information the latent captures, yielding lossy codes of global structure.
Editorial record
Plain-language summary
When a VAE's decoder is expressive enough to model the data on its own, the encoder has no incentive to put information into the latent variable. The authors show this follows from the cost of encoding the latent, then turn the effect into a design tool: by restricting what the decoder can see locally, they force the latent to carry global information such as shape while discarding texture. Combined with autoregressive flows for the prior, the model also set likelihood records at the time. The paper is a clear example of information-theoretic reasoning guiding architecture design.
Source record
Provenance
- Record ID
- P-678
- Record created
- 2026-09-26
- Last reviewed
- 2026-09-26
- Record version
- 1
- https://arxiv.org/abs/1611.02731
- arXiv:1611.02731
Citation caveat: Citation metadata is approximate and marked unverified in the source dataset.