Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing
Example of a 3-dimensional pdGMM with 2 components: a rank-2 component with support represented by the yellow plane, and a rank-1 component with support represented by the red line. Knowing the distribution on the open set E (in brown), intersecting both components, is sufficient to identify the pdGMMAbstract
Causal representation learning (CRL) aims to identify the underlying latent variables from high-dimensional observations, even when variables are dependent with each other. We study this problem for latent variables that follow a potentially degenerate Gaussian mixture distribution and that are only observed through the transformation via a piecewise affine mixing function. We provide a series of progressively stronger identifiability results for this challenging setting in which the probability density functions are ill-defined because of the potential degeneracy. For identifiability up to permutation and scaling, we leverage a sparsity regularization on the learned representation. Based on our theoretical results, we propose a two-stage method to estimate the latent variables by enforcing sparsity and Gaussianity in the learned representations. Experiments on synthetic and image data highlight our method’s effectiveness in recovering the ground-truth latent variables.
Type
Publication
In The Twenty-Ninth Annual Conference on Artificial Intelligence and Statistics
