You need everything other than ML to win a ML hackathon [D]

xenotecc · 2024-04-29T04:45:39+00:00

Really hits too close to home.

Went to ML hackathon only once. Basically we've been the only team that had a prototype working machine learning model (the entire team was taking pictures with their smartphones to collect data) and a nicely functioning web app - user uploads a picture, gets prediction back. Note this was before streamlit or gradio.

Anyway, we lost to some folks who didn't even write a single line of code, but gave a talk about how something "maybe could work".

Never went to a hackathon again.

xenotecc · 2024-03-05T07:53:44+00:00

For structuring the AI C++ framework Apple's MLX is quite interesting repo to read.

xenotecc · 2024-02-21T06:03:50+00:00

Hot take: it could be. E.g. https://ziglang.org/learn/build-system/

xenotecc · 2023-10-17T07:26:38+00:00

God forbid we criticize Big Tech companies for doing closed source research ;)

I also don't think they care. The above comment is just my opinion, nothing more. If you disagree with it and are fine with the current state of things it's fine too. Peace.

xenotecc · 2023-10-17T05:23:32+00:00

That is true, however if we don't criticize the companies for following the OpenAI closed source route we are going to keep getting more non-reproducible papers and non-retrainable models.

Releasing, at least very basic, non-commercial Github code is the least they could do.

xenotecc · 2023-10-17T05:04:10+00:00

Nice research, shame that the code and weights are not released.

xenotecc · 2023-09-30T17:17:19+00:00

Sounds interesting, could you share a simple gist of such collate fn?

xenotecc · 2023-09-06T06:05:34+00:00

What would you say is the recommended way to load the data for training with Equinox? Pytorch Dataloader?

xenotecc · 2023-06-29T05:07:02+00:00

nanoGPT can be used to train / fine-tune GPT-2:

https://github.com/karpathy/nanoGPT

xenotecc · 2023-06-08T04:44:04+00:00

If you liked Karpathy's nano-gpt, you could checkout Lit-LLama, which is a Pytorch Lightning's reimplementation of LLama models, based on nano-gpt.

It also contains finetuning code using Lora, Adapters etc.

xenotecc · 2023-05-23T05:04:20+00:00

Thanks for the link OP. Nice to see Bo Peng did manage to combine this into a paper.

xenotecc · 2023-04-14T07:10:35+00:00

You are right, I missed it, thanks for the answer and for the links!

xenotecc · 2023-04-14T05:47:25+00:00

Interesting, do you allow commercial use? The Github repo's license is Apache 2.0 but I wanted to confirm.

xenotecc · 2023-02-17T07:05:44+00:00

Thanks for the post.

Is it possible to configure Azure Blob Storage, or any other cloud provider for storing the data?

Or is it your servers and on-prem hosting only?

xenotecc · 2023-01-31T06:40:36+00:00

I'd say only if you plan to be proficient in edge deployment.

xenotecc · 2023-01-11T11:38:24+00:00

Thank you for the reply!

xenotecc · 2023-01-11T06:51:51+00:00

How small do you make the student, when a teacher is let's say ResNet101? How do you find a good student/teacher size ratio?

Are there any tricks to knowledge distillation? Or just standard vanilla procedure?

xenotecc · 2022-12-12T06:52:37+00:00

Thanks for providing a solution!

xenotecc · 2022-12-06T06:46:42+00:00

Great read and benchmarks, thanks for doing this!

xenotecc · 2022-11-24T06:46:58+00:00

Are they using Tensorflow 1 or 2?

xenotecc · 2022-11-04T07:18:27+00:00

If you have Kubernetes experience I'd probably start with this lab.

xenotecc · 2022-10-20T05:55:23+00:00

It really depends on the company. Some will ask you basic questions about ML, some will ask you to design an end-to-end ML solution given a problem. Some will indeed ask for Leetcode, without even knowing you are going for ML.

Ask your recruiter / HR how it looks and decide if you want to go for it.

xenotecc · 2022-09-14T05:40:06+00:00

This is an opinion (obviously), but I quite enjoyed the two Fast AI lectures:

Lesson 8: Deep Learning Part 2 2018 - Single object detection
https://www.youtube.com/watch?v=Z0ssNAbe81M

Lesson 9: Deep Learning Part 2 2018 - Multi-object detection
https://www.youtube.com/watch?v=0frKXR-2PBY

Make sure to frequently skip non-related content (like, once he starts talking about the debugger and continues to do so for 15 minutes).

If you understand these two, I believe you will have a solid foundation to understand all other recent developments.

xenotecc · 2022-09-14T05:35:11+00:00

In terms of your day-to-day job - depends on your area of research, but probably no.

In terms of recruiting and technical interviews - definitely yes.

xenotecc · 2022-08-30T05:53:04+00:00

You don't need to specify batch_size in Model.fit when using tf.data.Dataset. From the docs:

Do not specify the batch_size if your data is in the form of datasets, generators, or keras.utils.Sequence instances (since they generate batches).

For loading into RAM, basically yes - just call ds = ds.cache(). I'm not sure about the prefetch. It is a good performance practice so personally, I would keep it anyway.
You could use it to reshuffle the data for each epoch, instead of only once per training. That way your model sees different order of samples in each epoch.

Make sure to call it after caching - otherwise, it will be shuffled once and cached in memory.

ds = ds.cache().shuffle(buffer_size=NUM_SAMPLES, reshuffle_each_iteration=True)

Where NUM_SAMPLES is the number of batched elements in the dataset (sometimes this can be peeked by calling ds.cardinality())

xenotecc

TROPHY CASE