This research, which formed the core of my Master's thesis, introduced the Dalap (Discrete Analogue of Laplace) and Bitwise distributions — novel approaches that enable backpropagation through strictly integer-constrained domains. Dalap matches state-of-the-art results (PixelCNN++) in negative log-likelihood while maintaining differentiability, and Bitwise distributions outperform in high-entropy scenarios. The paper is currently under review at TMLR (arXiv: 2602.10751). The Master's thesis can be found here.