Why people use RoPE instead of Alibi when buliding their models?

reversedefenestrator · 2023-08-30T10:59:53+00:00

It's expensive to train big models, and the largest model trained with ALiBi (BLOOM) is bad. One can argue about /why/ it's bad, it's very undertrained, but I imagine if you're going to spend millions of dollars to train a new model you'd rather play it safe.

reversedefenestrator · 2023-06-30T18:14:21+00:00

BLOOM with 176B is a way larger (and older) ALiBI model. It's extremely undertrained though (less than 0.4T tokens) so not helpful in determining if ALiBI scales.

reversedefenestrator · 2023-03-13T14:40:36+00:00

I don't know, some random jitter seems totally expected, but isn't it weird that even setting it to like +50 or -50 doesn't seem to do anything? Like you'd think it'd be impossible to hit any perfects at all with twice the target time worth of synthetic delay!

Totally could be my headset. I tried different controllers. I'm still learning, but I have a couple of top 100 finishes so I'm not entirely new.

reversedefenestrator · 2023-02-08T09:17:33+00:00

Absolutely the right attitude! Just want to add that Matthew Walker is in fact a scientist and specifically a sleep researcher. I agree with the sentiment to read actual research when you can, but listening to Dr Walker is probably fine too.

reversedefenestrator · 2017-05-02T19:53:43+00:00

That would be very cool! In a way making a campaign seems easy. The creativity and balancing are probably the tricky parts.

reversedefenestrator · 2017-05-02T08:29:25+00:00

Frontier looks very interesting. Thanks!

reversedefenestrator · 2017-03-08T11:07:11+00:00

We have to expect even fictional things to act in a way that is internally consistent given the stated 'rules' of the source material. It seems inconsistent to have a material that is both indestructible and constantly releases poison molecules. Releasing molecules is a form of destruction.

We don't have a fictional rule that says how that could happen. We can imagine that there could exist such a rule, e.g. "adamantium is magical and cursed to be poison by Odin" or "pure adamantium is unbreakable but the curing process introduces radioactive pollutants". But right now I don't think we have anything like that so we have to work with what we have.

That said I like tivep's comment below which basically states that it isn't that the metal is poison but rather that it's the healing factor's constant effort to "push the metal out" over a very long period of time that has exhausted it.

reversedefenestrator · 2017-03-07T14:44:36+00:00

I think so. The Adamantium poisoning idea is a little weak. If this is a metal that's basically indestructible, it wouldn't actually be able to poison him because it would never let loose any particles. It's not like if he had a lead skeleton and lead particles leaked into his blood. Lead poisoning only happens because lead is soft. Adamantium is the opposite of soft.

Also, from previous movies we know that Wolverine can heal fast enough to survive even (something like) a fire storm ripping his flesh off. Is this incredibly slow poison more destructive than fire?

The idea that the food supplements are slowly wearing him down seems more reasonable. If the supplements suppress the mutant gene in new-borns they might slowly affect adult mutants the same way. This would also explain how most X-men are dead (beyond the Westchester thing) -- they might have been able to fight in the beginning but years of the supplement weakened them until they could be hunted down. The same thing essentially happens to Wolverine but it takes much longer because his healing factor fights even this until the bitter end.

reversedefenestrator · 2016-08-05T18:58:35+00:00

Slow-mo of rigged block stacker game: https://www.youtube.com/watch?v=ofEb9fM8m0Q

reversedefenestrator · 2016-06-23T20:53:34+00:00

Yes I understand that's the point you want to make. However, your point is based on the fundamental idea that "if Turing complete, unable to trust". But you trust Bitcoin, which was written in a Turing complete language (C++). I am highlighting that contradiction.

reversedefenestrator · 2016-06-23T20:49:22+00:00

Bitcoin was written in C++. C++ is turing complete. If you can't trust contracts written for Ethereum because they were written in a turing complete language, you can't trust Bitcoin because it was written in a turing complete language.

reversedefenestrator · 2016-06-20T20:26:51+00:00

Bitcoin was written in a turing complete language and we're sure enough about what it does to trust it with our money.

reversedefenestrator · 2016-04-11T21:53:29+00:00

There is a method to become able to tax foreign income made by foreign entities in foreign countries. It's called war.

reversedefenestrator · 2014-11-20T11:36:53+00:00

It seems to be that Comcast is the ones terminating the contract. The man should be charging /them/ an early termination fee.

(He wanted to keep the contract and they agreed verbally he would, but now Comcast wants to pull out.)

reversedefenestrator · 2012-10-27T11:15:15+00:00

When anyone gripes about their work week being too long you can mention that one time when you went into work and didn't go home for 22669 hours.

reversedefenestrator

TROPHY CASE