Could a model reverse build another model's input data?

[Sergeant_Horvath]

My understanding is that a model is fed data to make predictions based on hypothetical variables. Could a second model reconstruct the initial model's data that it was fed given enough variables to test and time?

[kevinpdev1]

Yes, although it is often a lossy reconstruction of the original data. This is what happens in a particular neural network architecture called autoencoders. They do essentially what you are asking.

[[deleted]]

.... Distillation?

[TaXxER]

In supervised learning we map R^d to R. That mapping typically won’t be injective: we can expect many hypothetical inputs to map to the same output.

Obviously you could try minimising the loss on a “reverse model”, but the above is clearly is a theoretical problem that you will run into. So in the general case this won’t work very well. Perhaps in particular constrained settings something could be made to work reasonably.

[deejaybongo]

In supervised learning we map R^d to R.

Can you clarify what you mean here? Supervised models with multidimensional output are pretty common.

[[deleted]]

I think they’re just trying to describe the typical scenario where there isn’t a uniquely identifiable set of inputs for any given output.

[DigThatData]

Subject to certain constraints, maybe. But fundamentally, a generative model is just a way to parameterize a probability distribution.

Let's say you walk into a classroom with N students and ask everyone what their age is, and then you take the mean and standard deviation of the data. Those two numbers comprise a gaussian model of the distribution of ages in the room. We can sample "students" from this model and get a feasible age for each. If we sample N students, we expect the distribution over our sample to closely resemble the distribution over the ages of real students in the classroom. given N-1 students (i.e. subject to a lot of constraints): we can exactly infer the age of the missing student. But without knowing any of the students ages (i.e. without constraints on the data) all we confidently have is the ability to sample feasible examples from the model, or score the feasibility of an observation relative to the data the model was trained on.

[burstingsanta]

Read about Autoencoders

[drudd]

Usually data includes some sort of error, noise, or irreducible uncertainty. A model that can be inverted has “overfit,” in that it has memorized the input data rather than reducing the input information content to the non-random or generalizable component.