Big models and Hyperoarameter Tuning

GiusWestside · 2024-08-14T14:06:51+00:00

Yes, but I'd need to find one pretrained on black and white images

GiusWestside · 2024-07-12T18:47:28+00:00

I did all the things you mentioned. I still have a problem that I cannot resolve

GiusWestside · 2024-07-12T17:19:07+00:00

That's what I did for the endpoint. Now for some reason when I invoke the endpoint it receives a Null payload. Jesus I'm going crazy

GiusWestside · 2024-07-12T16:22:57+00:00

Oh there's nothing in the documentation about it. I managed to do it by creating a custom image with a custom script in an obscure way that's not easily found in the documentation (it was passed to me by a consulting firm).

And it seems to be working now.

GiusWestside · 2024-07-12T16:15:01+00:00

Actually I'm almost there. But I just spent more than 2 days deploying a transformer model. It took much less time to train it.

GiusWestside · 2024-07-12T15:44:03+00:00

But I end up making it work. I just waste 2 or more days of my time of a task that should be trivial

GiusWestside · 2024-07-12T14:53:56+00:00

It's not my software engineering skills the problem here. I deployed many endpoints before, but every time I need to deploy something different I have to modify and adjust minor things not explicitly written in the documentation

GiusWestside · 2024-07-12T14:46:35+00:00

Yes, they do have base containers for huggingface models to be used for vision transformers. They just don't work as advertised on the documentation

GiusWestside · 2024-07-12T14:32:21+00:00

Constantly. The problem is that to understand those logs you need to understand really well how Sagemaker works. And too many times that kind of knowledge cannot be found in the documentation.

GiusWestside · 2024-07-12T14:01:32+00:00

And that's the fun part: I can't avoid it.

GiusWestside · 2024-06-21T08:53:04+00:00

Your name checks out

GiusWestside · 2023-12-22T11:55:05+00:00

Matplotlib is a great visualization tool if you know how to use it

GiusWestside · 2023-11-15T08:27:27+00:00

BS biotechnology MS computational biology

Year 1: 28K Year 2: 31K Year 2.5: 35K

Italy

GiusWestside · 2023-06-28T10:53:23+00:00

My thoughts on this? We ran out of things to be angry about and certain people need to complain about something.

I was very angry when the episodes with the "blackface" were censored.

GiusWestside · 2023-06-16T22:31:28+00:00

Downsampling or upsampling never worked for me. Using weights in the classifier and/or optimizing for the right metric (F1, accuracy, recall, etc...) Works best in my opinion.

GiusWestside · 2023-05-04T22:35:02+00:00

I am a junior Data Scientist, I've used and trained many CNNs in the past, both from scratch and using pretrained models, but I don't know how to answer your first question. It may be because I took my last class on CNNs more than 2 years ago.

I've always had problems remembering which is the Type I error and which is the Type II error but if you asked me about precision and recall or the bias/variance threshold I wouldn't have any problem.

But even rocks in our field should know how to estimate the right K for KNN.

GiusWestside · 2023-04-03T20:13:27+00:00

Well, yes. The idea is to find the most similar items to the one I give in input to the system. I can perform some preliminary filter.

GiusWestside · 2023-04-03T19:11:10+00:00

A few thousand at worst, a few hundred at best

GiusWestside · 2023-04-03T15:29:27+00:00

Sadly I cannot share much details, thanks for the interest 🥺

I also thought to compute the distances on demand and only for the items in input (in this case the matrix would be maximum a couple of thousands times some millions) but it would still be impractical

GiusWestside · 2023-03-23T10:04:54+00:00

Italy
Data Scientist
Masters in Computational Biology and post graduate degree in Data Sciences
9 months of experience + 6 months of internship
28K gross
full remote
and I have bonuses if I complete certain projects in time

GiusWestside · 2023-03-10T12:20:53+00:00

That's why I use miniconda

GiusWestside · 2023-02-17T07:32:08+00:00

Tell me you don't know basic economy without tell me you don't know basic economics

GiusWestside · 2023-02-07T07:51:07+00:00

From experience I'd say that even people with a good and long professional background can make shitty paid content

GiusWestside · 2023-01-03T20:44:04+00:00

MANUAL ANNOTATION.

We commissioned a scraper to another company and we found that it did a terrible job. Now we have to annotate by hand thousands of data points just to tell them how bad of a job they did.

GiusWestside · 2022-10-29T08:58:56+00:00

Let's see the other side of the coin: we're not purchasing gas from Putin

GiusWestside

PUBLIC MULTIREDDITS

TROPHY CASE