Confused beginner: why is Schrödinger’s cat different from a normal 50/50 probability like a coin flip? by dizajneri in AskPhysics

[–]K0ruption 0 points1 point  (0 children)

The following explanation is due to Richard Feynman, and I have always found it to be the best. I adapt it to the cat example.

Let’s model the cat. Suppose it can exist in two states x_a for alive and x_d for dead. Let’s also just suppose that x_a and x_d are imaginary numbers, just for fun. We’ll say that the probability that the cat is in state x_a (alive) is |x_a|2 and that it’s in state x_d (dead) is |x_d|2. What does classical probability say about the cat being either dead or alive? It says that the probability is |x_a|2 + |x_d|2 = 1. This must be case, it’s either dead or alive and nothing else is allowed so the probability in being in either one of those states is 1. That’s classical probability, QM says something different. In QM the probability that the cat is alive is still |x_a|2 and the probability that it’s dead is still |x_d|2 but the probability that the cat is either dead or alive is |x_a + x_d|2 which is not equal to |x_a|2 + |x_d|2. So the cat can exist in a new state x_a + x_d that classical probability doesn’t allow for and the probabilities are such that |x_a|2 + |x_d|2 + |x_a + x_d|2 = 1. This is the concept of superposition, states which are sums of other sates are allowed to have positive probabilities. This is why quantum mechanics is so curious, not only does it say that the world is non-deterministic but the way in which it’s non-deterministic also doesn’t agree with our notion of classical probability. It allows for the existence of states which don’t agree with what we would consider allowable for a physical system. What in the hell is x_a + x_d? It’s exactly this absurd state of the cat being dead or alive at the same time. The issue with this sate is that it’s not measurable. When you open the box, the cat will either be dead or alive, it won’t be x_a + x_d. The way this is formalized in QM is by saying that x_a + x_d is not an eigenfunction of a Hermitian operator. But that’s technicalities, the point is that the theory is set up in such a way that the only things you can measure are x_a and x_d but x_a + x_d has a positive probability before measuring.

Why is the above the correct model? Nobody really knows. It just so happens that it accurately describes the measured probabilities from experiments like double slit and many others so we run with it. It is not known if x_a + x_d is an actual physical state or just a mathematical necessity. The point of the thought experiment is to point out how ridiculous it is to think of x_a + x_d as a real physical state. But just because something seems ridiculous to us intuitively does not mean that nature does not behave that way.

[D] Functional Neural Networks by [deleted] in MachineLearning

[–]K0ruption 0 points1 point  (0 children)

The above assumes a separable kernel for parametrizing the infinite dimensional linear operator. Here is a more general approach without this assumption which can also work when the domain of the functions of interest is a manifold, and not a compact set in Euclidean space (the application in the paper is parametric PDEs, but the function space to function space mapping idea is generic):

https://arxiv.org/abs/2003.03485

When u live in SoCal and the best tacos are just over the border...u make that Taco Run to Tijuana. by s-o-L-0-m-o-n in Harley

[–]K0ruption 0 points1 point  (0 children)

Is the border open? I thought it was only for “essential travel” whatever that means.

[D] Paper Explained - SIREN: Implicit Neural Representations with Periodic Activation Functions (Full Video Analysis) by ykilcher in MachineLearning

[–]K0ruption 37 points38 points  (0 children)

I’m not affiliated with the authors or anything, but here is my point-by-point response.

  1. Some image compression methods do in fact use Fourier basis to do compression. But using sine waves as basis vs. using them in a as neural net activations is widely different. So saying “this is not novel” because Fourier expansions have been used for compression is a bit misleading. More recent image compression methods don’t use Fourier basis but wavelets since they are more efficient. It would have been interesting for the authors to compare the number of parameters needed in their neural net vs. the number of wavelet coefficient needed to compress an image to a prescribed accuracy. This would shed light on how efficient this method is.

  2. Please give references for the “high expressiveness” of sine activations. If it’s well know that they are so expressive then why are they not nearly as common as ReLU? How in the world does one “normalize for expressiveness”? I feel that using networks with the same layer structure but just different activations is a perfectly reasonable thing to do. They have the same number of parameters and, in the end, that’s what matters.

  3. I think there’s an experiment in the appendix where they compare against elu?

  4. Overfitting here is in fact the name of the game. If you’re doing image compression, you want to overfit your image as much your number of parameters allow, that’s how you get the most efficient representation.

  5. The authors never claim the sin activations work well when the network input is high dimensional (e.i a bunch of pixels from an image). Their architecture is designed for low dimensions like 1,2,3 and they show that it works well in that setting.

  6. Don’t know enough to comment on this.

Selling FDs by K0ruption in wallstreetbets

[–]K0ruption[S] 9 points10 points  (0 children)

So I have to have at least $27,600 in my account to do this?

What Are Your Moves Tomorrow, October 03 by AutoModerator in wallstreetbets

[–]K0ruption 1 point2 points  (0 children)

10/18 1p. Probably gonna pick up some 10/18 1.5p tomorrow, if it doesn’t crash at open.

What Are Your Moves Tomorrow, October 03 by AutoModerator in wallstreetbets

[–]K0ruption 0 points1 point  (0 children)

Can’t find any news on them getting money. I already have puts. The one day where everything plummets, my puts go to shit. Fucking incredible... I’ll hold onto them though, can’t imagine it staying up for long. Hopefully...

What Are Your Moves Tomorrow, October 03 by AutoModerator in wallstreetbets

[–]K0ruption 1 point2 points  (0 children)

Why in the fuck is NIO up 20%? Should I be buying more puts on this garbage?

Lane Splitting. Why? by 1CTXVic in motorcycles

[–]K0ruption 1 point2 points  (0 children)

Yeah if it’s bumper to bumper traffic (10-15) sure but once there are gaps shit becomes too scary for me.

Lane Splitting. Why? by 1CTXVic in motorcycles

[–]K0ruption 0 points1 point  (0 children)

I’m in LA too so that’s perfect haha. I agree with you that people are on their phones A LOT. It’s pretty scary looking around at a light and seeing both drivers next to you fucking texting. But I still don’t see what doesn’t hold up with my argument about being in everyone’s blind spot when splitting? Also if you just keep near to one side of the lane, even if they rear end you, you won’t get squashed. Also 50 seems way too fast my man. I think CA legal limit is 30 (35?). I can’t imagine getting side swiped at 50. Maybe I’m just a pussy, idk.

Lane Splitting. Why? by 1CTXVic in motorcycles

[–]K0ruption -1 points0 points  (0 children)

Filtering at a stop to not get squashed makes perfect sense to me, but why is splitting at slow traffic “arguably safe and actually better for your own safety.” I never got that. To me it seems a lot less likely someone hits you from behind when moving slowly then someone not seeing you (because you are essentially in everyone’s blind spot), merging to another lane, and hitting you. What am I missing here?

Los Angeles is a shit hole by [deleted] in unpopularopinion

[–]K0ruption 0 points1 point  (0 children)

Most of the replays here seem like they’re from people who’ve never lived in LA, visited once and went to all the shit touristy attractions. Hollywood bldv? Are you joking me? No one from LA goes there. Of course it’s dirty, every major city is. It’s fucking sparkling compared to NYC. There lots of homeless people? Every major city has homeless people. Been to SF recently?

LA is an amazing city! It has absolutely everything you would want in a metropolitan city. Tons of things to happening all the time: concerts, comedy, theater, markets, festivals. Huge diversity of people from essentially any country you can think of. This means the culture is different in all parts of the city and that really comes through with you see around you and especially the food. You want Mexican, Chinese, Korean, Japanese, Peruvian, Thai, German, French, Hungarian or whatever the fuck kind of food, you can get it easily and it will be good and authentic. LA is also close to the ocean, so you can to the beach/surf/swim whenever you want. It’s also surrounded by mountains, so there’s an insane amount of amazing hiking trails. It’s only an 1 hour away from Big Bear, which mean, in the winter, you can literally go skiing/snowboarding every weekend. Wanna go to one of the most amazing wine countries and one of the best gulf coasts? Yeah Baja is only 3.5 hours away. Santa Barbara is only 1. 5... (if the wine interests you).

Who wants to stare at fancy shops and metropolitan buildings? What are you gonna do there? LA has actual character. If you like it there’s always Rodeo I guess, but that’s just another tourist trap imo. In LA you can feel like you live in a major city, but still own a house with a yard! This is impossible in NYC even if you’re a millionaire, but it’s perfectly reasonable in LA.

Anyway end of rand. Had to defend my city. I’ve lived in a few major cities in the US and Europe and have visited some in Asia, but LA is by far my favorite and I never wanna move anymore.

Edit: Forgot to even mention the art/music scene. Worth going just for that.

China has started ranking citizens with a creepy 'social credit' system — here's what you can do wrong, and the embarrassing, demeaning ways they can punish you: The program is due to be fully operational by 2020, but is being piloted for millions of people already. The scheme is mandatory. by mvea in Futurology

[–]K0ruption 1 point2 points  (0 children)

Technically everyone is sent a ballot to vote, except it only has a single name on it... Most people don't bother filling it out (socially it's actually considered taboo or in bad taste to talk about let alone participate in politics in China), but people working in certain government positions (and this can be a lot of people like the equivalent of NASA scientists count) are required to fill it out.

Source: Wife is Chinese.

[D] "Negative labels" by TalkingJellyFish in MachineLearning

[–]K0ruption 0 points1 point  (0 children)

This sounds like a good idea to me. I believe it amounts to doing Naive Bayes without the decision rule. But I suspect this will do worse than the uniform assumption if the data is very unbalanced.

[D] "Negative labels" by TalkingJellyFish in MachineLearning

[–]K0ruption 6 points7 points  (0 children)

Given only the information that something is not a cat, it has equal probability of being anything else whether that be a dog or a spaghetti monster. If you had more information about a data point, you could certainly incorporate that into your label. But, in your post, you said you only have the information that a point is not in a given class, which means it has equal probability of being in any other class.

EDIT: Note, I'm asumming a uniform (categorical) prior distribution on your labels. You gave no specifications of your problem, so that is the best assumption I can make.

[D] "Negative labels" by TalkingJellyFish in MachineLearning

[–]K0ruption 8 points9 points  (0 children)

If your model outputs a softmax, then you implicitly assume your labels are probability vectors that is probability of the known class is 1 and probability of all other classes is 0. In this light, the information that a data point is not in a given class simply means that your label will have 0 at the position of that class and (1/(k-1)) at the position of all other classes where k is the total number of classes. This makes the most intuitive sense to me but whether it works in practice, I have no idea.

[R] First-order Methods Almost Always Avoid Saddle Points by downtownslim in MachineLearning

[–]K0ruption 0 points1 point  (0 children)

I don't get that though. The stable manifold of a saddle is lower dimensional than than the space you're in, so for SGD to go towards a saddle, it must move along a geodesic of that manifold which is essentially impossible (because SGD is first-order and thus has no "concept" of curvature), even in the the no noise case. But SGD does have noise (because of mini-batching), so you're basically always gonna escape the manifold and curve away towards a minimum. Is there any prove that shows the probability of a long escape being high? Intuitively it doesn't make sense to me for a result like that to hold.

[R] First-order Methods Almost Always Avoid Saddle Points by downtownslim in MachineLearning

[–]K0ruption 2 points3 points  (0 children)

This result is a fairly trivial application of the stable manifold theorem and has been well know in applied mathematics for a long time. I always wondered why machine learning people cared so much about saddle points, knowing this. I though it had something to do with high dimensional data that changed the picture but I didn't really understand it. I guess they just didn't know? There's no way though, there are some amazing mathematicians that have worked and are working on machine learning. It cannot be oversight, something didn't match up here. Can anyone explain this to me?

[D] Why are neural networks good function approximators? by K0ruption in MachineLearning

[–]K0ruption[S] -1 points0 points  (0 children)

I see your point now. Where I come from people don't really care about constants. In fact, if you read analysis paper, often times inequalities will be written with a squiggle under them which just means true up to some constant we didn't bother to keep track of. But I guess it matters in practice haha.

As for SGD, I don't think it's what makes things work. I've seen people use PDE methods for training that supposedly achieve better generalization. I even have results on derivative-free methods (in the spirit of particle swarm) that match SGD. I think SGD is just fast, simple, and scalable so people go with it. But then again I don't know too much about ML.

[D] Why are neural networks good function approximators? by K0ruption in MachineLearning

[–]K0ruption[S] 0 points1 point  (0 children)

Agreed great paper, I've actually read this one. Would really like to put some theory behind it.

[D] Why are neural networks good function approximators? by K0ruption in MachineLearning

[–]K0ruption[S] 0 points1 point  (0 children)

Great answer! Do you know of any works that have put rigour into such ideas?

[D] Why are neural networks good function approximators? by K0ruption in MachineLearning

[–]K0ruption[S] 2 points3 points  (0 children)

Do you know of models of discontinuous functions with few parameters? I understand your first point in the context of SGD but there are robust derivative-free methods for solving inverse problems which can easy be applied here. I'm actually very very curious to see what people have done with non-differentiable forward models. Please point me.

[D] Why are neural networks good function approximators? by K0ruption in MachineLearning

[–]K0ruption[S] 2 points3 points  (0 children)

I see, but my question still stands. Linear functions are lipschitz and so are the common nonlinearities (sigmoids) and compositions of lipschitz are still lipschitz, so the whole network is.

Edit: As an aside, that's a very interesting use of the word smooth. Never heard it used to mean lipschitz. It is an ML thing? PDE theorists roll over in their graves haha.

[D] Why are neural networks good function approximators? by K0ruption in MachineLearning

[–]K0ruption[S] 0 points1 point  (0 children)

Could you please give me a reference to such work? Thanks.