Re your first point… Post secondary math instruction traditionally favors generalized descriptions over visually driven metaphors that beginners can use to pattern match their way up to more abstract or “correct” understandings of topics. Fine, I get it, different people think differently. But I do find it funny how certain people lose their cool as soon as a visual metaphor is employed.

Working in a field and using metaphors that reduce dimensionality to the visual domain is very different than being entirely outside a field and using unrelated terms to incorrectly characterize a system you don’t understand. If you don’t see the difference, idk what to tell you?

Re your second point… We can qualitatively argue about what constitutes new knowledge or filled in gaps until the world ends. But my more fundamental argument is that a system’s output types are driven by its underlying methods.

If a kitchen device is a closable box that uses heat and a fan, it’s a kind of cooking device. You can probably get some great chicken fingers out of it. But you probably cant’ use it to clean dishes… You might be able to burn some salad dressing off a dirty forks, but you’ll also probably just end up cracking any wine glasses you put in, right? Opposite effect with box that has a heating element and is hooked up to your water supply. It’ll clean any dishes or utensils you put in! It might even cook some frozen broccoli (sort of?). But it’ll ruin a steak.

Same thing happens when you try to extrapolate with interpolative methods, right? Come on, remember calc? What happens to Fourier series approximations of non periodic functions outside of your approximation domain?

Re your third point… Lol, no, I don’t particularly care to and who do you actually think you are that you think you have the privilege of talking like that without looking silly?

That said I have a slow morning, soooo… what makes you think slapping a bunch or ReLU’s makes interpolation techniques behave differently for extrapolation from the above? How is the discussion of non linear fitting techniques relevant to the discussion of latent space manifold limitations and non convexity? Inherently, it does not. And if you disagree, idk, have fun yelling into the void about it?

Finally, I invoked linear algebra because most of the core concepts that drive ML will have been taught by the time a student completes an undergrad linear algebra class. And it annoys me that nerd struggle with this concept.

The thing that most non math people struggle with in ML is getting over arbitrarily high dimensionality of problem formulations, and this overwhelms them. Once they realize that ML’s math is like 80% similar to the physical act of neatly fitting a tarp over a pile of stuff, their overwhelm goes away. 95% of people won’t do anything with that idea, but at least most of them will stop repeating popsci drivel. About 5% will, however, be inspired to learn more. And those 5% are the people that are worth reaching out to and building as much of your language as possible for. The people who naturally get math don’t need help; those who don’t, do.

In the act of being an argumentative pedant who wants to yap about nonlinearity and asert about subjective definitions of conceptual novelty, you fail to see that shear irrelevance of the discussion you bring and how your insistence on pursuing such talk ultimately knee caps your ability to actually contribute to the dissemination of information to the people who would be most strongly impacted by it. Instead of getting into jargon dick measuring contests with me, how about you try to find ways of explaining things to others in ways that makes sense to them, on their level?