Real-world datasets characterized by discrete features are ubiquitous: from categorical surveys to clinical questionnaires, from unweighted networks to DNA sequences. Nevertheless, the most common unsupervised dimensional reduction methods are designed for continuous spaces, and their use for discrete spaces can lead to errors and biases. In this Letter we introduce an algorithm to infer the intrinsic dimension (ID) of datasets embedded in discrete spaces. We demonstrate its accuracy on benchmark datasets, and we apply it to analyze a metagenomic dataset for species fingerprinting, finding a surprisingly small ID, of order 2. This suggests that evolutive pressure acts on a low-dimensional manifold despite the high dimensionality of sequences' space.

Intrinsic Dimension Estimation for Discrete Metrics / Macocco, Iuri; Glielmo, Aldo; Grilli, Jacopo; Laio, Alessandro. - In: PHYSICAL REVIEW LETTERS. - ISSN 0031-9007. - 130:6(2023), pp. 1-6. [10.1103/PhysRevLett.130.067401]

Intrinsic Dimension Estimation for Discrete Metrics

Macocco, Iuri;Glielmo, Aldo;Laio, Alessandro
2023-01-01

Abstract

Real-world datasets characterized by discrete features are ubiquitous: from categorical surveys to clinical questionnaires, from unweighted networks to DNA sequences. Nevertheless, the most common unsupervised dimensional reduction methods are designed for continuous spaces, and their use for discrete spaces can lead to errors and biases. In this Letter we introduce an algorithm to infer the intrinsic dimension (ID) of datasets embedded in discrete spaces. We demonstrate its accuracy on benchmark datasets, and we apply it to analyze a metagenomic dataset for species fingerprinting, finding a surprisingly small ID, of order 2. This suggests that evolutive pressure acts on a low-dimensional manifold despite the high dimensionality of sequences' space.
2023
130
6
1
6
067401
https://arxiv.org/abs/2207.09688
Macocco, Iuri; Glielmo, Aldo; Grilli, Jacopo; Laio, Alessandro
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.11767/131772
Citazioni
  • ???jsp.display-item.citation.pmc??? 0
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 5
social impact