"Asymptotic Analysis of Lattice-Based Quantization"

Peter W. Moo

Ph.D., Electrical Engineering: Systems
University of Michigan
August 1998

Advisor: D.L. Neuhoff


Lattice-based quantization is an attractive method of vector quantization, because of its potentially low complexity and its ability to form nearly spherical cells. In this dissertation, three lattice-based quantization methods are optimized using high resolution analysis.

In the first part of this dissertation, fixed-rate lattice quantization for a class of generalized Gaussian sources is considered. Asymptotic expressions for the optimal scaling factor and resulting minimum distortion are presented. These expressions are derived by minimizing upper and lower bounds to distortion. It is also shown that for scale-optimized lattice quantization, granular distortion asymptotically dominates overload distortion, and the ratio of optimal lattice quantizer distortion to optimal vector quantizer distortion is shown to increase without bound as rate increases.

In the second part, multidimensional companding, which utilizes a nonlinear compressor function and a lattice quantizer, is optimized for memoryless sources. Under certain technical assumptions on the compressor function, it is shown that the best compressor function consists of the best scalar compressor functions for each component of the source vector. The gain of optimal companding compared to scalar quantization is shown to be the gain in normalized moment of inertia of the lattice cell compared to a cube.

The third part considers uniform polar quantization, which utilizes a polar transformation and a two dimensional integer lattice quantizer. It is shown that the optimal rate allocation between magnitude and phase gives increasingly more rate to the magnitude as total rate increases, compared to nonuniform polar, where the difference between optimal magnitude and phase rates is a constant. In addition, a unified analysis of several nonuniform polar quantization schemes is developed by focusing on their point densities and inertial profiles and using Bennett's integral to express the mean-squared error. The subsequent analysis is straightforward and leads to new insights into the relationship between polar quantization and Cartesian quantization.