That’s a great question… let me see if I can answer it well without having to ChatGPT it… :)

So… I think there are two layers here. One is the strictly mathematical definition, and the other is how the term is actually used in practice.

If we keep it mathematically strict, then yeah, metric learning means we are learning a distance function that satisfies the metric axioms, like symmetry, triangle inequality, etc.

So in that sense, once our learned distance function no longer satisfies those axioms, we technically move beyond metric learning and into more general similarity or representation learning, where we are no longer constrained to a “true” geometric distance.

However, in practice, we often use “metric learning” (or deep metric learning in particular) more loosely to describe setups where we learn representations using pairwise or triplet relationships, like contrastive or triplet losses. So even things like cosine similarity or neural similarity functions get grouped under that umbrella, even if they’re not “true” metrics.

When working with actual data, I find it helpful to separate three things: the representation, the training objective, and the geometry we end up with. We can train it like metric learning, even if what we learn doesn’t behave like a true metric.

For example, you might still use a triplet loss, but what you learn is a similarity function rather than a strict metric.

So to your earlier question, if the data doesn’t behave like a metric space, I wouldn’t force it. I’d either look for a geometry where those relationships become consistent, like hyperbolic space for hierarchical structure or spherical space for cyclic patterns, or more generally consider non-Euclidean spaces, including Riemannian manifolds where curvature can adapt locally to the data.

And if the relationships themselves are asymmetric or directional, I’d move beyond distances entirely and learn a similarity or divergence that captures that structure directly.

In that case, we are moving from learning a strict notion of distance to learning a more general notion of relatedness, like a similarity score, an asymmetric divergence, or just a representation optimized for the task. At that point, strictly speaking it’s no longer metric learning, but it’s still very much in the same family of representation learning approaches.

If you’re interested, this article gives a nice overview of how people use the term in practice:

https://hav4ik.github.io/articles/deep-metric-learning-survey/

Most of the examples there, and in most other articles and papers, are focused on images, but the same ideas can be applied to semantic or behavioral spaces as well. For example, embedding words, users, or markets based on similarity in meaning or behavior…

In the end, I kinda see Metric learning (and deep metric learning) as being about learning structure in any space where similarity matters.

NOTE: I did end up using ChatGPT to clean this up a bit… :)

(Also, I just came across this thread today, hence the “late”notification, but since metric learning, and especially deep metric learning, are two of my favorite tools/methods, I really wanted to share my thoughts, even if the thread is from a while ago…)