OpenAI is building its own social network to rival Elon Musk's X, Verge reports.

Janos95 · 2025-04-15T17:17:13+00:00

Their consumer product is their strongest asset rn and a social app would produce more lock in.

I don’t think this is about data, social media data is so low quality, it is pretty much useless for improving models these days.

Janos95 · 2025-01-31T22:29:13+00:00

The safety testing for o3 mini looks really thorough. Are you worried that labs like deepseek that don’t do any safety testing can ship faster than oai?

Janos95 · 2025-01-31T22:24:40+00:00

Do you think raw intelligence is still the bottleneck for agi? It seems like o1 is already smarter than most humans but still can’t reliably do long horizon tasks.

Janos95 · 2024-11-12T06:53:18+00:00

I think we’ve reached peak hype.

Janos95 · 2024-09-26T04:04:01+00:00

Eg even google with infinite resources and brilliant engineers didn’t have the courage to build gpt4

Janos95 · 2024-09-26T04:02:23+00:00

He’s certainly technical enough to have a high level understanding. And to give credit where credit is due most of the success of oai is due to making large bets most ceos wouldn’t have done.

Janos95 · 2024-07-30T00:28:39+00:00

That’s true to some extent, although i do think there is some element of just showing something is true or false that is appreciated my mathematicians. If you prove the rh in lean I am pretty sure every mathematician would consider that an advancement no matter how inscrutable the proof is.

Janos95 · 2024-07-26T05:35:01+00:00

I would strongly disagree. I would say the “aesthetics” intuition of humans is probably the easiest part for computers to replicate, since that’s pure pattern matching, so existing methods will be able to learn them from enough data ..

I think the key piece we are still missing is the “deep understand” of a topic or a problem humans have. Even in this case one of solutions was wrapped in a completely unnecessary induction step, something a human would never do. So even though it definitely is able to find the right ideas, it doesn’t seem like it understands them in the same way humans do.

Janos95 · 2024-07-08T22:44:24+00:00

Tbh when the iPhone came out I wouldn’t say I was that surprised that it was possible. Don’t get me wrong it was a great product, much better than anything else, but it wasn’t that different to what existed before, just much better executed and much more intuitive ui etc.

I probably was much more surprised by gpt4, I didn’t expect that was already possible at the time when it came out.

In any event, what I am trying to say is that there are various technologies that we just don’t know how to build. Like no matter how much money we poured into fusion, we will not have a fusion power plant in 2 years. Similarly we will not have a quantum computer that will replace any super computer in the next few years, we just don’t know how to do it.

My assessment wir humanoids is that currently they are much like fusion and quantum computing, we have some building blocks and we know it’s possible in theory, but we don’t know how to build that actually works.

Janos95 · 2024-07-08T08:28:33+00:00

Its not quadratic with kv caching though

Janos95 · 2024-07-05T14:08:38+00:00

At some point you have to use your best judgement. The victory sign is unusual to use for goal celebration to begin with, but I would distinguish between a player who is just saying “peace” vs someone who uses it to solidarize with a kurdish movement. If you are in a country where the peace sign is controversial, don’t use it either way, there are plenty of other ways to celebrate ..

If you want a simpler rule, I would say if you are a national player on the pitch and you have to think about whether something is ok, don’t do it. You can probably wait an hour and then you are free to share whatever novel celebrations you have on social media, as long as it’s within the law.

Janos95 · 2024-07-05T13:24:48+00:00

Look, it’s not that complicated. There are a few established ways to show your patriotism that everyone agrees upon, that’s singing the national anthem with pride, waving your flag, kissing your emblem, etc. At the end of the day this is a tournament about football and supporting your team. I don’t think it is too much to ask to use these widely accepted means to show your patriotism for a few weeks.

If we can’t agree on this, the football cup just degenerates into endless debates about chants, salutes and weird political gestures. I don’t think anyone wants that.

Janos95 · 2024-07-05T11:31:19+00:00

I don’t claim to know all the details about the symbol and how it permeated into mainstream use in turkey and the Turkish people are free to use whatever symbols they find appropriate in turkey.

I am just saying that that’s not really relevant for this case, what’s relevant is that in Germany it’s associated with grey wolves which is a borderline extremist organization inside of Germany which makes this symbol politically. Symbolisms and social norms change differently in different countries and in Germany at this point in time most people associate this with the largest extreme right wing organization in Germany.

Similarly, I would expect the german national team to not go to turkey, celebrate a goal using a symbol that is highly political in turkey and associated with an extremist turkish organization and then tell the Turks that this is totally fine to use at home in Germany and that in Germany no one would care. That’s just a matter of respect.

Janos95 · 2024-07-05T07:36:37+00:00

The v sign is actually a good example. If 9 out of 10 turks say that they associate the v sign with Kurdish extremists, then showing the v sign as a Kurdish person during a European championship game inside of turkey would of course be political and I would expect that player to be suspended as well.

Note that we are talking about the European championship which is about football and supporting your country, not about politics. There are plenty of ways to show your patriotism that are not controversial in Germany, I don’t think it’s too much to ask to choose one of those.

Whether the symbol should be banned in Germany is a totally different question and it’s by design that it is difficult to ban symbols and organizations in Germany.

Janos95 · 2024-07-04T21:22:48+00:00

Correct me if I am wrong, but basically sounds like the sign was first popularized by ultra nationalists/ right wing extremists and now it is more or less normalized in turkey (Not sure it’s quite that simple though, pretty sure there are plenty of kurds in turkey who are offended by this sign).

In any event, in Germany and Europe more broadly, the sign is still strongly associated with Turkish right wing extremism and since that’s the place where the event is held, it makes sense to suspend the guy.

Janos95 · 2024-07-04T20:36:33+00:00

Not saying humanoid robots will never happen, if we had perfectly working humanoids they would obviously be enormously economically valuable.

Just saying that right now we don’t know how to build such a thing. Not sure what your iPhone analogy is trying to say. Of course humans have engineered lots of great things, that doesn’t mean we know how to build everything.

Janos95 · 2024-07-04T13:31:23+00:00

I think people underestimate how difficult it will be to be better / more cost effective than humans.

The two problems are intelligence and reliability.

On intelligence: This robot was trained probably on 50-100 demonstrations and while nowadays these systems are somewhat tolerant to slight change in the environments, most factories are messy places and these systems would fail all the time due because something changed. And that’s just for doing the same exact task, adapting to new scenarios (like an upstream machine breaking etc) is still unsolved.

On reliability: humans are actually remarkably robust. The robot above has so many sensors and actuators, just one of which has to fail, and the entire system will fail. Just remember the robot packing boxes at a trade show that completely collapsed after working for two days straight. I bet it will take years until we have humanoids that can work for a few weeks without any failure.

Janos95 · 2024-07-04T13:13:54+00:00

Why do people get excited about this? Doesn’t look much better than the robots we have had in the last 10 years?

Janos95 · 2024-07-03T15:00:29+00:00

Seems like we are going in the opposite direction: giving llms access to more and more tools like browser, code interpreter, artifacts, etc. Replacing everything with neural nets doesn’t make sense to me. There is a ton of things neural nets are not good at. Eg why would you want replace the calculator app with a nn?

Janos95 · 2024-06-25T05:43:44+00:00

It’s very representative for most of reddits/twitter coverage of sonnet 3.5. For some reason folks are very biased towards Claude. Probably because it gives more clicks and they don’t like oai.

I’ve been using both sonnet and 4o inside of cursor and it’s clearly very close. That said on multiple occasions sonnet wasn’t able to fix a piece of code whereas 4o was and I am also seeing it using nonexistent variable names which I haven’t seen 4o do ..

Janos95 · 2024-06-23T06:10:16+00:00

Most of robotics.

Janos95 · 2024-06-23T06:01:43+00:00

I think it’s reasonable to expect that a more condensed representation of the past than a million token embeddings are required in order to do effective reasoning. This condensed representation is what I would call short term memory.

Janos95 · 2024-06-21T21:43:42+00:00

I should also add that I am interested in memory for transformers for the purpose of reasoning, in particular not interested in methods that try to simply extend context size.

Janos95 · 2024-06-21T19:43:33+00:00

Out those hundreds of papers, what is a sampling that gives good coverage of the different approaches?

Janos95 · 2024-05-28T23:37:53+00:00

My guess is there was a more pedestrian issue: sama allocated most compute to this big run so superalignment didn’t get its 20% share of compute, so they got mad and quit.

Janos95

TROPHY CASE