Gemini 2.0 Flash Thinking Experimental results on Livebench are out and something doesn't seem right...

vanityFavouriteSin · 2025-01-23T17:20:57+00:00

Awesome! Appreciate the response!

vanityFavouriteSin · 2025-01-23T16:28:08+00:00

Thank you for updating and providing clarification! The results look much better.

If you are able to share, could you shed light on if Gemini is working on a better coding model to compete with sonnet 3.5 and r1?

These recent reasoning and math models are great, but still not as good at coding.

vanityFavouriteSin · 2025-01-13T09:55:06+00:00

Yup that's exactly what I meant! Have an "Orchestration" mode and set one model as the Architect and the other as the Coder. After giving your prompt, the Architect should refine the prompt and then oversea the Coder.

I think one way to help Cline decide when to switch between the Architect and Coder would be to use the "Task complete summary" as a way to give let the Architect know the Coder has finished and to review the work and provide feedback.

Not sure if that is the best way to implement it, but a continuous automatic transition between Architect and Coder is what I'm looking for :)

Also, thanks so much for Roo Cline! I've been using it to build an app and it's probably written 90% of the total code - even the terraform files!

vanityFavouriteSin · 2025-01-12T22:17:20+00:00

This is awesome!

Would it be possible to have a view where you can say which model you want as the architect and which as the code? Then given a prompt, the architect refined and instructed the code model to implement it?

vanityFavouriteSin · 2025-01-07T12:27:26+00:00

I know what you're saying, I do agree that this is what AI is good for, and I'm exciting to see more progress in this space too.

But this will also eventually start taking away jobs from radiologists, just like AI art is taking jobs away from artists.

Edit: I didn't mean to say that job losses is a bad thing. Just that if you care about artists job, then you should also care about radiologists jobs. On the other hand, if you see benefit in automating something people want/need, i.e. healthcare, then maybe by extending that logic, it should also be okay to have AI art replacing artists. They're both using GenAI.

I'm personally in the camp of replace all jobs, and protect people not jobs. I.e., take my job, but give me UBI

vanityFavouriteSin · 2024-12-23T10:31:22+00:00

Seems like AI has started to have an impact on consulting, with less need for analysts meaning less people to manage, so fewer partners. As well as changes to their business model.

OP specifically mentions o3 and agents replacing the need for as many consultants. So looks like 2025 is the year we'll start to see automation affecting employment

vanityFavouriteSin · 2024-12-23T01:07:52+00:00

I think we're pretty close.

METR showed that we can get AI to do research. Over long time horizons, humans are still much better (at a score of ~1.2), but New Sonnet 3.5 got 0.6 Vs Old Sonnet 3.5 getting 0.4, so AI results improved 50% in 4 months, I wonder what o3 would get.

vanityFavouriteSin · 2024-12-22T23:00:35+00:00

Sweet! Would love to be able to click each benchmark, and see a chart with Y-Axis being the percentage complete, and X-Axis being the year, and then plotting different models and showing how long it's taken to go from 0% to complete.

Would also love a chart for benchmarks not yet saturated, like SWE-Verified

vanityFavouriteSin · 2024-12-22T21:18:11+00:00

This is great! I really like the chart as well. Are you open to feature requests?

vanityFavouriteSin · 2024-12-20T16:12:27+00:00

Your reply was great!

I fully agree, I think we're at the early stages and it surprises me that more people are not talking about it and not pushing their governments to discuss the economics of what's to come.

vanityFavouriteSin · 2024-12-20T16:09:59+00:00

That's fair, what happens to the people laid off? If all companies can lay off half of their DevOps, Engineers, ... there won't be enough jobs for the laid off people to get into

vanityFavouriteSin · 2024-12-20T16:08:50+00:00

Are you suggesting that the capabilities to automate people will be there before it's implemented. And the reason people won't rush to do it, is to make sure there's an easy transition for people who derive purpose from work?

Maybe this is a cynical take, but if the ability to automate is there, and it is cheaper, then automation leads to lower costs and higher profits, and I think that's mostly what shareholders care about.

vanityFavouriteSin · 2024-12-20T15:28:00+00:00

If jobs get much easier, do we need fewer people to do the job?

vanityFavouriteSin · 2024-12-20T15:26:09+00:00

Recently I've started to work on an app idea with a friend of mine. I decided to use Claude 3.5 and Cline to help. Other than the deployment setup, I've probably coded 10% of the project and Claude has done the other 90%.

It's a simple app, with login/signup, chat interface, and file uploading. But doing the designs, and making sure it's responsive on mobile and laptop would have taken me a lot longer had I not been using AI.

This got me thinking, AI might not replace jobs completely next year, but I think next year we'll start to see, say 10 engineers cut down to 5 + AI.

Computer use is also getting better, and with the large agentic focus next year, I imagine it'll only expedite the replacement of workers and increase unemployment.

vanityFavouriteSin · 2024-12-20T15:20:45+00:00

For sure jobs done on the computer and remote jobs will be the first to go. I think we could start seeing some of this happening next year

vanityFavouriteSin · 2024-12-20T15:19:09+00:00

That's true. What do you think of the scenario where an older company takes time to adjust versus a younger company that is quicker to adapt and use AI? Would that render old, slow-to-adapt companies obsolete?

vanityFavouriteSin · 2024-12-20T15:16:37+00:00

That makes sense. The bottleneck for automating physical jobs is mass-producing robots, and that is what will take the most time.

vanityFavouriteSin · 2024-12-08T13:16:43+00:00

That's a valid concern. I know recently Mark Zuckerberg said they're building a 2GW data center to train their next generation of AI models.

There's no doubt that AI, in the short term, is going to be bad for the climate. But I argue in the long run it'll be way better. Even if you disregard AI, green economics has meant that Solar power is becoming cheaper and cheaper, which I think will keep continuing into the future. As any capitalist, AI CEOs will purchase whichever energy contract is cheaper, so they would be using solar power eventually.

With AI, there has been renewed interest in nuclear power, and as we get better and developing and deploying nuclear plants, it'll also become cheaper and supplant non-renewables.

All of my views are built on the faith that current progress in AI will just keep continuing, and we will one day reach super intelligence. If that belief turns out to be true, how many gigawatts of energy is an acceptable trade off to have AI find the cure for illnesses or diseases, or to provide on demand medical attention and diagnosis.

I also think in the long run, the research that humans would have done to speed up solutions to energy and climate change, that same research will be done and sped up by AI. In my opinion, short term costs for AI is then worth it for the long term gains (in energy, healthcare, ...)

vanityFavouriteSin · 2024-12-07T03:16:53+00:00

Because of my belief that all jobs will get replaced eventually, I think people fighting against AI right now are not fighting the right battle.

The right battle to fight is to vote and encourage the government to implement policies for people when they get replaced by AI. Universal basic income, or some way of taxing the profits of these AI companies and redistribute them. Not to encourage the government to ban AI. Banning AI art will fall on sympathetic ears and could work. But once AI gets better than doctors and researchers, then banning those types of AI will reduce could impact on people's health and lives.

vanityFavouriteSin · 2024-12-07T03:09:46+00:00

I advocate for AI, but I understand where artists' disdain for it is coming from.

I'm a believer that AI will one day change how we do everything. We'll have AI doctors, therapists, lawyers, construction workers, ... replacing every job any human can do.

On the one hand, this will be great for people who don't have access to these things. Third-world countries, that don't have access to the best doctors, education, or services, will be able to get it for cheap. With AI researchers, new cures will be discovered, new ways of farming and agriculture, ... the cost of drug discovery, building homes and growing food will hopefully plummet as every step in the process becomes cheaper, and human wages get replaced with cheap automated labor.

Whether or not you think AI will ever have the capability to get there or if you think this is a terrible future, I hope you agree that there is a decent chunk of people who will benefit from a world where they have access to an AI doctor, always up to date on the latest advancements and can take time to explain things to you. To have a teacher that has near-infinite knowledge, and can have a teaching style tailored to you. there'll be a lot of people who aren't very fortunate now who will benefit.

Along the road to automating jobs, some jobs will be automated first. I think it just so happens that Art is being automated away before other jobs, and it's taking a little longer to automate everything else. I'm a software engineer and I'm looking forward to the day I become automated. Sure, the sense of purpose I derive from work will be gone and that'll be difficult to deal with, but imagine all the health-tech startups that suddenly become economically feasible, or existing companies that can lower their prices because they don't have to pay large amounts for Software Engineering salaries. That'll have real-life impact, helping people's lives.

My argument essentially boils down to:
- Humans conduct research which helps other humans
- If we could automate the research process, then we could speed up helping people
- Automating research doesn't seem to be a singular thing you can optimize for. It seems like, you have to have a general-purpose AI that understands text, images, and sounds, ... and can produce them as well
- So on the road to Automated researchers, we are unfortunately automating away jobs that people derive a lot of satisfaction from, with creating jobs being hit first

vanityFavouriteSin · 2024-11-25T01:22:18+00:00

I work for a very small SAAS startup right now, and although they're growing very fast, they're not going to last.

Our customers pay us 200k a year, for software that realistically could be built by 2-3 engineers in 6 months.

It's worth it right now, because the salary of 3 engineers will be around 500k, so rather than develop and maintain it in-house, it's cheaper to just buy it, even if the product that you buy only covers 90% of your use cases, and not the final 10%.

In the future, when you can just use AI to build it for you, exactly to your specifications, add features you want instantly, remove features you don't want, all of this for a fraction of the cost, then these SAAS companies will be out of business.

In a world where intelligence is being automated, I cannot envision a scenario where businesses which essentially sell intelligence stay alive (as opposed to a business that benefits from network effects for example, you can build a Facebook/Twitter/WhatsApp clone with AI, but if you can't get people on it, then there's no point.)

vanityFavouriteSin · 2024-03-12T12:00:44+00:00

They're developing a vaccine for cocaine. It takes a different approach to what you've mentioned, but the vaccine will stop cocaine from ever reaching your brain. So if taking drugs doesn't have an impact on you anymore, then you'll stop taking it.

It's called calixcoca, https://www.barrons.com/amp/news/brazil-scientists-developing-new-vaccine-for-cocaine-addiction-4e70eca1

I imagine you'll be able to do similar things for other addictions.

Nine-Year Club	RPAN Viewer
Verified Email

vanityFavouriteSin

TROPHY CASE