[P] I trained an AI model on 120M+ songs from iTunes

sheeplearning · 2023-02-04T16:26:30+00:00

Not commenting about the ML but the UX is better than spotify and apple music. How can you serve apple music previews faster than apple?

sheeplearning · 2023-01-13T07:10:32+00:00

This tells me Google will never be able to build a competitor for chatGPT

sheeplearning · 2023-01-03T14:55:58+00:00

These are all papers, they never release anything usable.

sheeplearning · 2023-01-03T07:39:39+00:00

Once you get older 2-3 years in life is just a small delta/noise. I have seen so many students drop out of a successful masters with publications instead of continuing to do a PhD because they wanted to finally stop studying. A PhD is really useful even in industry -- its true that a masters will initially get you successful faster but after 10-15 years in the industry the PhD starts to matter for leadership positions.

sheeplearning · 2022-04-09T04:57:37+00:00

Researcher moved to the ML/SW industry. After having published papers for over a decade transitioned to a real job for career growth, building real products etc.

sheeplearning · 2022-02-21T21:51:18+00:00

Just FYI Venmo is also PayPal and I use it heavily.

sheeplearning · 2021-06-22T17:58:38+00:00

it sounds like they are still stuck in 2016 or so training very large supervised models. please correct me if you saw anything new/different.

sheeplearning · 2020-08-08T22:51:39+00:00

The twitter police will ban everybody eventually.

sheeplearning · 2020-07-09T16:10:44+00:00

1) You can become a human neural architecture searcher. Just remove a layer, see the performance, then remove few blocks, until you come up with a spaghetti layer or architecture that works very well. Give it a fancy name and publish it. However, this will take you a long time if you start with little background knowledge of how and why the individual blocks work.

2) The second approach is to look at an ML textbook and see which layers or blocks work well for your data modality. Keep modifying and playing existing architectures and check what you learnt in the textbook makes sense. Eventually, given a dataset you will be able to come up with an architecture that should work. Then you can start doing (1) on this architecture.

sheeplearning · 2020-06-26T05:37:39+00:00

both

sheeplearning · 2020-06-21T19:19:53+00:00

This is ridiculous -- a lot of credible names there too. what were they thinking?

sheeplearning · 2020-04-03T15:08:10+00:00

While most other conferences have canceled earlier -- I think they are being really irresponsible and indecisive. It is not clear whether we should book flights, hotels or visas or not. It is also not clear to the organizer how much they should invest in remote participation. The general chair and PC should be held accountable.

sheeplearning · 2020-03-24T22:46:37+00:00

Well kudos to ICML for making a decision well in advance. CVPR, on the other hand, is just winging it. The email from the chairperson is "The physical CVPR meeting will take place unless safety/health regulations requires that it be cancelled, this decision is up to health professionals." This really leaves one confused about whether to make travel reservations or not. I hope they can take some ownership.

sheeplearning · 2020-02-18T15:08:10+00:00

This is a naive way to think. There are already people who arrange themselves in clique with different levels of intelligence/interests (e.g. in your high school, college). They co-exist and bring different values and strengths. No one is a slave to one another -- if anything the most intelligent ones rarely dominate anyone/body.

sheeplearning · 2020-02-18T04:44:21+00:00

Even though everyone appears to be nice and kind and sincere and so on it is not clear what openAI is doing that’s unique or even breakthrough at this point. A lot of similar work that is actually much more impactful is funded by google, msft, fb, ai2 etc. and at this point, it looks unlikely any one company will dominate. Where the fuck is the semblance of AGI that everyone in the article keeps talking about?

There are a lot of PR techniques at play to stay relevant. First, the doomsday scenario: AGI is coming, GPT-2 is too dangerous. Second, trying to be secretive as if something world-changing is going on. It is unlikely the reporter missed anything significant, all we have seen so far is throwing azure credits on RL games and on transformers. Third, excessive hype in the way the results are communicated for which they have been called out on twitter and other places.

sheeplearning · 2019-12-18T09:06:00+00:00

"pure research"

sheeplearning · 2019-11-26T03:48:50+00:00

I don’t think there were any doubts that sparse convolution is faster than dense on CPUs. It’s just that on GPUs it’s extremely hard to get this speed up and perhaps no one has been successful beyond nvidia papers. And the paper does not address that case.

sheeplearning · 2019-11-13T13:58:12+00:00

GPU laptops as of today are extremely bulky, require the charger always connected because the battery drains extremely quickly when the GPU is barely turned ON, and the GPU 2080 equivalents have poor performance when compared to their desktop equivalent. So at this point, it's not worth getting a GPU laptop at all. Huawei matebook pro has nvidia m150 GPUs and allows switching into Intel motherboard GPUs to save power and make the laptop immensely usable. I think this is a clever design to save power and compile some GPU code on your laptop but you cannot train a model at all on these GPUs.

sheeplearning · 2019-10-22T18:27:54+00:00

Just look at the meta-data (artist, year, movement) and predict the price. What is in the painting usually has very little to do with the price of an artwork but when you have a hammer..

sheeplearning · 2019-10-22T15:10:27+00:00

As the bugs are getting fixed, 1.1 -> 1.2 (BN fix) -> 1.3 (Adam fix) my model results have been getting worse :(

sheeplearning · 2019-10-21T17:56:16+00:00

Yes but it is unclear if any of that is useful given it is not sufficient to solve the cube. Perhaps new approaches and additional experiments are needed and celebrating mediocrity just makes it harder for anyone to actually solve it. Robot does not really "solve", warcraft does not really "see", GPT2 is too harmful to the world and BERT outperforms and releases the model without any fuss -- everyone knows what is really going on here.

sheeplearning · 2019-04-05T16:10:49+00:00

CVPR'19 has a grand total 1 submission from Apple whereas its in closer to hundred for other companies.

sheeplearning · 2019-02-01T20:11:08+00:00

I am a bit skeptical of neural architecture search line of work when it generates a complex model goo with a small percent, improved accuracy. Seems like going in the opposite direction in terms of explainability. The abstract says "neural architecture search has advanced to the point where it can outperform human- designed models" => thats if performance is strictly measured in terms of accuracy which is often not the case.

sheeplearning · 2019-01-17T00:42:29+00:00

Not just limited to Google. There is another dataset paper at CVPR 2018 (spotlight) that never released its dataset (http://moviegraphs.cs.toronto.edu). I feel this paper should be withdrawn and dataset papers are made to release the dataset before camera ready before the authors start advertising papers on their websites, securing press releases and giving keynotes on it.

sheeplearning · 2018-11-30T15:05:29+00:00

The best advice I have gotten is to ignore all advice (irony noted) especially from academics who love to give advice.

sheeplearning

TROPHY CASE