Leland McInnes, John Healy, James Melville
2018 · arXiv
UMAP (Uniform Manifold Approximation and Projection) is a novel manifold learning technique for dimension reduction. UMAP is constructed from a theoretical framework based in Riemannian geometry and algebraic topology. The result is a practical scalable algorithm that applies to real world data. The UMAP algorithm is competitive with t-SNE for visualization quality, and arguably preserves more of the global structure with superior run time performance. Furthermore, UMAP has no computational restrictions on embedding dimension, making it viable as a general purpose dimension reduction technique for machine learning.
Metadata: arXiv. Abstracts retain their original authors’ rights.
Original HN comments, with labels from simple text rules. A label describes the whole comment and may miss context or irony. Read the discussion to make up your own mind. About the labels
5 selected excerpts · 5 total mentions
The closest I know of is the work on UMAP. I interviewed Leland McInnes who explained to me in detail how category theory was a big part of helping him connect the dots, even though the final result does not strictly need it in the actual code. Given the relative improvement over the previous state of the art (t-SNE), it's the only example that really makes me reconsider my poo-pooing the way category theory is discussed in software. https://arxiv.org/abs/1802.03426
Check out UMAP [1]. It's an alternative to t-SNE for dimension reduction that's grounded in topological category theory. [1] https://arxiv.org/abs/1802.03426
UMAP[1] and t-SNE[2] are two very similar and popular ways to map a set of points in a high dimensional space to a low-dimensional (usually 2D or 3D) space by assuming all the points are close to some smooth curved surface (a manifold[3]). These are unsupervised non-linear dimensional reduction techniques; if you're familar with linear dimensional reduction techniques like PCA[4] you can think of UMAP and t-SNE as curved generalizations of that. t-SNE is widely known to be hard to interpret and unstable with respect to arbitrary hyper-parameters[5] and I would assume UMAP suffers similar problems. It's also extremely difficult to define "similarity" between documents and…
authors. Techniques like LSA[6] and word embeddings[7] exist but are still an open area of research. Based on similar concerns as yours, I doubt any of these issues were seriously addressed when this visualization was developed. [1]: https://arxiv.org/abs/1802.03426 [2]: https://en.wikipedia.org/wiki/T-distributed_stochastic_neigh... [3]: https://en.wikipedia.org/wiki/Manifold [4]: https://en.wikipedia.org/wiki/Principal_component_analysis [5]: https://stats.stackexchange.com/questions/263539/clustering-... [6]: https://en.wikipedia.org/wiki/Latent_semantic_analysis [7]: https://en.wikipedia.org/wiki/Word_embedding
If what you're asking about is the math, the steps are (essentially) as follows: 1. A Riemannian manifold is constructed from the dataset. 2. The manifold is approximately mapped to an n-dimensional topological structure. 3. The reduced embedding is an (n - k)-dimensional projection equivalent to the initial topological structure, where k is the number of dimensions you'd like to reduce by. I don't know how well that answers your question because it's difficult to simplify the math beyond that. But you can also check out the paper on arXiv. [1] The underlying idea is to transform the data into a topological representation, analyze its structure, then find a much smaller…
(dimensionally speaking) topological structure which is either the same thing ("equivalent") or very close to it. You get most of the way there by thinking about how two things which look very different can be topologically the same based on their properties. A pretty accessible demonstration of that idea is the classical donut <-> coffee mug example on the Wikipedia page for homeomorphisms. [2] __________________ 1. https://arxiv.org/pdf/1802.03426.pdf 2. https://en.wikipedia.org/wiki/Homeomorphism
structure to the space of prime factors There is topological structure https://math.stackexchange.com/questions/2879/mapping-natura... http://ncatlab.org/nlab/show/arithmetic+topology as well as geometric (arithmetic geometry). Question is how does the picture correspond to any of it. Glancing at the UMAP paper https://arxiv.org/abs/1802.03426 it looks like a proper invariant preserving embedding, so maybe loops in the image are real. Even if not, it does hint artistically at sorts of stuff that exist.
More from the same topics.
William P. Thurston
In response to Jaffe and Quinn [math.HO/9307227], the author discusses forms of progress in mathematics that are not captured by formal proofs of theorems, especially in his own work in the theory of foliations and geometrization of 3-manifolds and dynamical systems.
Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan +11
Artificial intelligence (AI) researchers have been developing and refining large language models (LLMs) that exhibit remarkable capabilities across a variety of domains and tasks, challenging our understanding of learning and cognition.
Martin Kleppmann
The CAP Theorem is a frequently cited impossibility result in distributed systems, especially among NoSQL distributed databases.
Tom B. Brown, Benjamin Mann, Nick Ryder +28
Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task.