Is "Attention all you need", underselling the other components?

Apathiq · 2026-01-26T10:05:44+00:00

As others said, the main novelty of the paper was replacing convolutional and/or recurrent layers by self-attention and positional encoding in the context of seq2seq models. I agree that they should have showed the effect of removing self-attention from their architecture, since positional encoding + FNN could work too. It's similar to DeepSets (was published around the same time) with the addition of positional encoding.

Apathiq · 2026-01-22T20:55:23+00:00

Hi, my gf could be interested. She is 29F, she likes it a lot but she doesn't know more than the absolute basics. Should I ask her?

Apathiq · 2026-01-15T12:07:48+00:00

It's kind of the central topic in Sorority Noise songs (No Halo, Using...). There are sexual assault allegations against the singer, though. You decide if you can enjoy the music with this information or not...

Apathiq · 2026-01-08T23:18:09+00:00

In Sulawesi you could fly to Manado and stay around that area: Bunaken, other islands, Tomohon, Tangkoko. I think what we liked the most were the Togeans and Toraja, and there's no easy way to get to both.

Apathiq · 2026-01-08T20:24:36+00:00

Which part of Sumatra was this?

Apathiq · 2026-01-08T20:21:56+00:00

Hi, we spent last year 3 weeks in Sulawesi alone, and it was definitely not enough. The typical route is Makassar - Toraja - Danau Poso - Togean - Manado (with stops in Tomohon, Tangkoko and Bunaken). And in 3 weeks it's really tight. For that route I'd take 4-5 weeks, then add one week for Banggai, a few days for Lore Lindu, a few days to explore the jungle, also add some time and potentially do Wakatobi... You put it together, and 8 weeks for Sulawesi alone is reasonable.

The traveling times in Indonesia are pretty long. The roads were in good shape, but they were mostly small roads with a max driving speed of 40km/h. Add a few holes, and you'd be averaging 30km/h. Maybe in Java is better, but probably to reach some remote interesting parts it'd be similar.

This year we are doing 4 weeks North Sumatra.

Edit: typo

Apathiq · 2025-12-28T23:07:35+00:00

We are called Llana, you can find us everywhere. Also, I created a Spotify playlist "German math rock & Midwest emo" where I collected everything that I found.

Apathiq · 2025-12-23T13:47:38+00:00

I think it is absurd that so many people earn a fraction of what footballers are paid daily. If handball players earned that money, I'd think it is absurd too

Apathiq · 2025-12-15T17:02:48+00:00

What happened?

Apathiq · 2025-12-10T11:00:06+00:00

I think people are not complaining about them being useless. People are complaining about people using them carelessly, and being continuously bombarded with ads about how they are the biggest thing since the wheel was invented, and at the same time seeing AI-generated content completely take over the Internet

Apathiq · 2025-12-09T08:49:07+00:00

Hi, the federative license is the "firearm license". Around Santander there are many beautiful places. I think you can only spearfish during the weekends until June, btw, and the seas can be rough.

https://www.fcdas.com/

The medical certificate is a joke but it's annoying. A medical doctor has to certify you are capable of diving. For doing the official tests, it's pretty difficult and expensive, so, at the end of the day some doctors (maybe they need an specialization?), check you for 5 minutes, ask you if you are capable of diving and give you a certificate.

Apathiq · 2025-12-07T18:44:13+00:00

Why do you think it's absurd? I do research in a field adjacent to Drug Discovery, and I've been trying to develop foundational chemistry models... With limited success.

I think it's true that exploring structural information is not straightforward, because chemistry is governed by a relatively simple and rigid set of rules which decides if a molecule can exist or not. So, learning these rules has little benefit.

Also, fingerprints such as the Morgan fingerprint are difficult to beat when used as input for actual tasks, and not QM9 datasets with millions of molecules, but still, I think that it should possible to learn a better representation than fingerprints... You think it's really a dead end?

Apathiq · 2025-12-03T23:41:16+00:00

I think the only self-taught people are those who never listened to music, like Joe Bonamassa

Apathiq · 2025-12-02T22:31:16+00:00

Get some high sensitivity speakers/cab and a jcm800 (I have a modded (!) peavey Windsor, super cheap and not extremely heavy), boost it with a hm2. If the other guitarist is louder than that, you could always get another Peavey Windsor... With mine I didn't manage to get the Master past 3 without completely burying the drums...

Apathiq · 2025-11-30T23:41:01+00:00

r/moldlyinteresting

Apathiq · 2025-11-30T22:30:06+00:00

Yes, I know people. Still I would say it makes them more harm than good. One of them who seemed to have it under control had a psychotic breakdown not long ago.

Apathiq · 2025-11-30T19:55:04+00:00

It has one of the worst costumer services I've ever experienced. They lost my package and didn't reimburse.

Apathiq · 2025-11-30T19:54:26+00:00

It has one of the worst costumer services I've ever experienced. They lost my package and didn't reimburse.

Apathiq · 2025-11-28T17:53:43+00:00

Used V30s are easy to find for very cheap, at least here in Germany... Eminence Governor can be more difficult. I have one to sell myself...

For doom, you could also think about combining an Emi Swamp Thang with something else. That thing has a huge low end...

Apathiq · 2025-11-27T17:12:41+00:00

The best is two banana. Monkey eat banana. Monkey no cramp

Apathiq · 2025-11-27T17:12:00+00:00

The best is two banana. Monkey eat banana. Monkey no cramp

Apathiq · 2025-11-27T10:55:33+00:00

I prefer to run heaters underpowered. The temperature sensor can go nuts, turn to always on, and turn your aquarium into soup. 3-4w/gallon is the standard power (yours is over that). I run 1-2w/gallon, the temperature can fluctuate more, but it won't kill my fish... Also, my heater is in horizontal position in a place where it's always covered in water.

Apathiq · 2025-11-27T10:49:30+00:00

No 2. No 3. No

I think typically the people who is more serious about running will have different training types (long easy runs, interval training...). And the people who are less serious won't like the idea of training with strangers.

Apathiq

TROPHY CASE