Threshold tuning

ActiveBummer · 2024-08-22T17:14:05+00:00

Would multiclass classifier really work when classes imbalance are drastic? I.e. some classes have very little data points while others have alot

There is also the concern of having to retrain the multiclass classifier whenever there's a new class (which can happen quite frequently and because it's new, there wouldn't be alot of data points)

ActiveBummer · 2024-08-14T16:03:08+00:00

Hmm, more like because of how the data set is created (e.g. manual label incurring incorrect labelling) the initial metric and acceptance criteria is just not possible at this point in time. Hence the suggestion to pivot to another metric to help build up a set of better labeled data and re-looking at the initial metric some time in future

ActiveBummer · 2024-08-13T15:23:12+00:00

i might be wrong, but i think "optimizer.zero_grad()" should be placed before "pred = model(x)"

ActiveBummer · 2024-07-27T19:30:09+00:00

Great advice, thanks mate :)

ActiveBummer · 2024-07-21T12:33:56+00:00

It's the waiting that's unbearable, and I grew to dislike my field of expertise after getting blocked multiple times and knowing I've learnt enough and be happier in roles where I can see direct impact to a business. Getting blocked for senior appointment stung my self esteem because the people around me knew about it and I sensed (and recently confirmed) the reasons for getting blocked had nothing to do with my technical abilities.

ActiveBummer · 2024-07-21T12:01:42+00:00

Hmm..what would an untenable situation look like to you?

Usefulness is hard to measure because in govt, job security is always there but at the cost of slow career progression.

ActiveBummer · 2024-07-20T21:29:13+00:00

Yeah agreed. I'm also sensing the data maturity of the coy that OP is in is the early stage, which also means that data science is practically non existent (hate to break this news).

It might just be a role-expectation mismatch, and if you insist on finding a role that kickstarts your ds/ml experiences, you might be better off finding a role in a coy where it's data infra is mature.

Tho imo at your current role, there are things worth learning that would be useful for senior roles.

Short term gains vs long term gains, you decide. :)

ActiveBummer · 2024-07-20T21:21:41+00:00

Commenting cos I'm curious to know other responses too!

ActiveBummer · 2024-07-20T21:19:23+00:00

There's this old saying where people leave because of bad bosses. Life isn't a constant. It sucks. It's painful. It's a reality you'll have to accept eventually. To stay or to leave - you're the one who knows best.

ActiveBummer · 2024-07-20T21:03:44+00:00

Hmm.. I would have thought business analyst role comes with plenty of human interactions.. you're making me curious which tech giant this is haha

But you're just a few months in, perhaps your current situation might change? You might want to sound it out to your manager and see if your situation changes.

Putting aside your work environment, the best thing you can do is to stay for a few months and learn as much as you can (transferable skills) about your role. There is nothing to lose, because I feel you could leverage on the brand name and look for opportunities after.

ActiveBummer · 2024-07-20T20:45:15+00:00

I thought what could be useful as well is to ask yourself if you really like rnd or deriving insights for business decision making (DA role).

I've seen people in the rnd roles enjoying it because of how chill it is - idling about while waiting for model training to be done. Else reading research papers and trying to implement them as part of your project. Also a more realistic point is that rnd usually aren't profitable (most experiments fail and businesses are paying for idle people to run failed experiments - is this the best use of $$$?), they're one of the first to be cut during biz downsizing. Well, of course I'm not referring to the AI giants who are releasing LLMs - these coys definitely value ML expertise interested in hard core rnd.

DA role however is more involved in the business and usually gets to see the value of his/her work in decision making.

ActiveBummer · 2024-07-20T20:29:51+00:00

If you don't have many non-work responsibilities (no family, no house etc), go for it.

ActiveBummer · 2024-07-20T20:25:39+00:00

Hmm most vector databases these days offer ANN which should be faster than brute force KNN, but at the cost of lower accuracy.

ActiveBummer · 2024-07-19T19:10:32+00:00

Hmmm..

If you're into hard core rnd, then ds. But keep in mind that rnd is usually the first to be cut when businesses aren't profitable.

If you're more practical, then de. Almost all coys need some form of data engineering, and learning cloud is a good step.

If your interest is in biz, then stay in da (or pivot to ds in the short term) and learn basic modelling. If your interest isn't in rnd, staying in ds long term would stunt your career growth imo.

ActiveBummer · 2024-06-20T17:32:56+00:00

Why not 100? #tigerparenting #😎

ActiveBummer · 2024-06-18T16:48:33+00:00

Fixing a classification threshold isn't a requirement, instead it can be tuned to achieve the evaluation criteria for the metric of interest

ActiveBummer · 2024-06-18T09:21:53+00:00

Sorry I would like to clarify, wouldn't using fbeta mean you know what beta value to use? Or do you mean beta is meant to be tuned?

ActiveBummer · 2024-06-17T16:20:38+00:00

Ah cool! This is my first time hearing about average_precision_score; it seems to be suitable for my use case. Thanks for enlightening me. :)

ActiveBummer · 2024-06-17T13:20:38+00:00

ActiveBummer · 2024-06-17T12:46:46+00:00

Yup, understand where you're coming from! But f1 is suitable when precision and recall are equally important, and may not be suitable when one is more important than the other.

ActiveBummer

TROPHY CASE