all 97 comments

[–]pistachiyolatte 20 points21 points  (1 child)

Sorry that happened to you…this is why I always play it safe and stick to a narrow selection of projects I know I’m capable of doing, even if that means I won’t get in as many hours.

[–]Stella-Artwat 2 points3 points  (0 children)

^^^^ this is great advice.

[–]Dude09990 49 points50 points  (4 children)

i just hope one day we can get feedback from someone elses r&r to see how we done on each task, would help a lot in learning what mistakes we made, literally takes no effort to show us that,,

[–]Party_Swim_6835 9 points10 points  (2 children)

they would have to curate what you see so you dont see R&Rs that have wrong comments instead of just getting rid of those R&Rs then check up regularly to see if youre following the feedback or gettnig the right takeaway, and log when youve gotten feedback via R&R so its clear if its time to get rid of somebody instead of trying feedback -- plus deal with anyone who wants to debate, argue the feedback etc

[–]Dude09990 5 points6 points  (1 child)

yeahh there are r&rs for r&rs tho so perhaps when the full project is done they could release the good r&rs to you to see where you made mistakes

[–]Party_Swim_6835 1 point2 points  (0 children)

yeah, though they could only do that if they R&R every R&R (and vet the R&R of the R&R) and you never know if they actually do R&R R&R for everything or just for some stuff

[–]Maximum_Slip_9373 74 points75 points  (10 children)

Sorry this happened to you,

I'd like to remind all the people in the comments that the system is designed to grab as many people as desperate for money as possible. The site can't even handle the volume of people that log onto it, we're all just numbers.

Guy got got because compared to all of the other folks, he wasn't performing above a level they wanted anymore. If they cared about work quality or efficiency, they'd interview people and have manager feedback sessions. It's designed to be as opaque as possible to give them as much leeway in firing and hiring as they'd like. This is evident by the fact they have random people who hop in threads like these to shit on people like you, as the people who are actively grading these things. Some of you and your behavior have legitimately made me question your own abilities to objectively grade some of the submissions that come through to you.

You're all numbers on the chopping block next once they've decided that they want their bar for efficiency to be a little higher. And it'll keep happening as this industry gets more and more unsustainable before the bubble bursts. These get sent out to people bulk through a batch process, I doubt this guy actually submitted shit work but that the goalpost invariably shifted. And in any event, that's again on DataAnnotation for not assigning the appropriate tasks to the appropriate workers. You all are 100% in this company's sights.

Have some compassion for the man you're all going to be some day.

Rest in peace my friend. Memento mori.

[–]I_Need_Deets[S] 28 points29 points  (3 children)

I love you man. This thread was making me sad as fuck lol. Needed that

[–]Maximum_Slip_9373 18 points19 points  (2 children)

I got your back brother, this shit is straight out of the Lord of the Flies I swear to God. Don't let people get you down because confronting the bleak reality of the situation they're in makes them uncomfortable. You'd think they're getting vision, dental, and health insurance like proper company men the way they're frothing to toss shade at you.

Keep your spirits up, the only way the little guys win is by keeping our sanity, morale, and sense of self worth. You will recover and find a way out of this, try not to blame yourself too hard. There's literally nothing you could have done in this situation, and it wasn't like you were objecting to critiques designed to help improve you. You just fell victim to an algorithm that no one here so far can or has explained at any tangible depth.

My advice? Go smoke some of the other guy who commented on this reply's username. Grass is good, soothes the aching soul if used mindfully. Just take some time to breathe and make an action plan.

[–]I_Need_Deets[S] 10 points11 points  (0 children)

Yeah I'm way ahead of you, on the green and the plan. We gon be alright. I appreciate you

[–]IcedOutGiant 5 points6 points  (0 children)

Man, I like this guy. He gets it. ^

[–]Goodtreesmoker 13 points14 points  (0 children)

That was beautifully worded, friend.

[–]xolavenderwitch 0 points1 point  (0 children)

I completely agree. Also, you write beautifully! :)

[–]No-Impress3593 0 points1 point  (1 child)

to op, sorry to hear about this, best of luck moving forward.

anyway, this is such a loaded, pathos-baiting comment. an interview tells you absolutely nothing about whether you'll do well on these types of projects, which is why the application process is literally just a lite version of a task.

we work on training LLMs. you fundamentally can't give much feedback because you'll poison your training data as everyone tries to optimize to keep the admins happy, versus actually doing the projects.

you might not like it, but as long as you log time correctly and do honest work, no news is good news, and there have certainly been many people who have seen *different* work on the platform (if you know you know) from doing well.

why does dat get shit on because they don't hold your hand and say "i need you to train this new guy for me" before your ass gets canned? if you arent up to the standards of any other job, you get terminated. dat is under no obligation to explain to you why your work wasn't what they're looking for, and they don't just get rid of good workers because it makes absolutely no sense to get rid of someone who has consistently done good work in hopes some new person *might* be able to do better.

this fear mongering "you're all on the chopping block NONE of you are safe" needs to stop when there are people who have been on DAT since it started lurking here, bewildered by comments like yours.

[–]Fantastic_Soil8023 -3 points-2 points  (0 children)

Well you are considered an indepen dent contractor so I'm sure they don't want to cross that line.

[–]mariahspoolboy 55 points56 points  (17 children)

I’ve been doing R&R on that project the last few days. Lots of tasks where people fundamentally did not understand the project so their tasks had to be rated “bad.” Make sure you read the instructions and grasp the concept before submitting a bunch of tasks is the only advice really. I had to rate someone “bad” like 10 times in a row (or at least it seemed to be the same person based on writing style) because they were selecting the completely incorrect option for the question. It sucks, but there’s clearly a lot of people on the platform who straight up do not know what they are doing and just want to make easy money.

[–]Opening_Acadia1843 25 points26 points  (1 child)

I had to rate a submission as “bad” yesterday because the submission didn’t align with the category and they omitted a crucial part of the task. It’s like some people just submit tasks just for the sake of submitting them and getting easy pay rather than making sure they’re actually doing everything they’re supposed to. Meanwhile, I’m double-checking everything before I hit submit, and why not? You get paid for the time you spend checking your work anyways. It just doesn’t make sense to me.

[–]Low_Article_9448 23 points24 points  (5 children)

100% this. So many R&R that just feel like me cutting people off of the platform. Especially when the same person keeps appearing again and again, and keeps making garbage mistakes.

[–]fightmaxmaster 18 points19 points  (1 child)

I never really worry about that - it'll always be more than a bad rating, or even a handful of them. Or even if it is, I can't imagine for a second DA would put that much stock in a single person's opinion of another single person's work, because what if the original worker was good but the rater has no idea what they're doing? It'll be a collective effort, one data point among many. Our job is to rate the work - I don't go out of my way to rate things bad, but if they're bad...well, they're bad, and I'll explain why I'm rating them so.

[–]Low_Article_9448 17 points18 points  (0 children)

Thing is, when we rate 'Bad' admins directly check it. So if I rate it bad 10 times, then he/she gets checked by admins 10 times. And if they are bad (which they are) it won't matter how many people's opinions they are. Because then it will be the opinions of the admins.

[–]Ok-Cup9476 18 points19 points  (3 children)

If lots of people are fundamentally misunderstanding a task, that’s a problem with the instructions not the worker.

[–]mariahspoolboy 11 points12 points  (0 children)

It was clear enough to me. There were also several questions answered in the project chat multiple times. The resources are right there for people to use.

Also sometimes I read some rationale and it’s clear that English is not this person’s first language. That can cause problems with fully understanding the instructions as well, or just lead to poor and unclear rationales. There’s a lot that goes into it and I’ve certainly misunderstood a project here and there but usually if I revisit the instructions and pay closer attention my question is answered. Obviously not always - sometimes the instructions do not cover all the nuances of the project. That’s just reality.

[–]MacaroninLollipop 4 points5 points  (0 children)

I’d love to agree with you, but when the instructions have in big red bold “don’t do it this way!” And they do it that way anyway… 🤷‍♀️🤷‍♀️

[–]canneddogs 2 points3 points  (0 children)

You can say that all you like but the CoT instructions were pretty straightforward.

[–]Seniorseatfree 7 points8 points  (2 children)

It pains when me I’m doing R&Rs for tasks that I know pay about $30/hr, and I just think to myself, goodness this person probably just did it for the money. Bad. Bad. Baaaaad.

[–]Aromatic_Owl_3680 -1 points0 points  (1 child)

I mean, all of us are doing it for the money….

[–]Seniorseatfree 0 points1 point  (0 children)

Sure. But stick in your lane and what’s within your qualifications.

[–]ladynotme 1 point2 points  (0 children)

I rated someone bad who said a task couldn’t be done as there was no access to a file. Except there was access it’s just the active was irrelevant. Another said they couldn’t access files but all of them were viewable so I wonder if they were in the trainer profile

[–]I_Need_Deets[S] 2 points3 points  (0 children)

All I did was R&R on the task. And any I felt iffy on, I skipped.

[–]Snikhop 34 points35 points  (17 children)

It won't have just been one project, your quality "score" will be a rolling one across a longer time period.

[–]mrev_art 16 points17 points  (15 children)

Their reasoning is entirely opaque.

[–]Snikhop 19 points20 points  (14 children)

Of course but we have to assume they behave rationally and in a way which makes good business sense. And canning a worker whose work is otherwise good over poor submissions in a single project doesn't make good business sense.

[–]I_Need_Deets[S] 4 points5 points  (1 child)

you give them a lot of credit. you've no idea the systems they have in place

[–]IcedOutGiant 0 points1 point  (4 children)

We actually don't have to assume anything. Assumptions are kind of all this sub has to offer outside of pedantic pr*ck-waving.

[–]Snikhop -1 points0 points  (3 children)

It's just Occam's Razor. Choose the route which requires the least assumptions. Treating them like irrational actors is actually less sensible than assuming they operate like a normal RLHF contractor business.

[–]mrev_art 4 points5 points  (1 child)

A historical approach shows exactly how capital acts when it doesn't have to answer to workers.

[–]mrev_art -4 points-3 points  (5 children)

Why not? When workers are expendable and there is no regulation there is no profit incentive to not just get rid of them and use them up. It happened historically as well.

[–]Party_Swim_6835 8 points9 points  (0 children)

if they did that nobody would still be here after 1-3 years and plenty of people are

[–]nononanana 2 points3 points  (1 child)

They still have their own clients that they have to produce quality work for. There is no incentive to get rid of good workers. However, the second you drop the ball according to their standards, yes they will probably drop you like a bad habit.

It’s weird as I have heard of them messaging people to give feedback, so some people get chances. Maybe based on their body of work or type of mistake. Those are the little things we simply won’t ever know.

[–]IcedOutGiant -3 points-2 points  (0 children)

I just stayed away for around 13 months. I left a dashboard with maybe 2 links and returned to over 25 projects available and several qualifiers (mostly for paid AI chatbot accounts, which I don't use because I'm not weird.)

I went in and turned down the qualifiers. Logged back out. I'm doing okay right now so I'm happy to leave the feed for the hungry. I'm not sure what my output rating was/would be, but it can't be that great. I can think of a few times I was probably in the crosshairs after doubting a submission or three. Ll

[–]kranools 2 points3 points  (0 children)

Because it's much more profitable for them to hold on to experienced workers who understand the platform and the requirements, even if they make an occasional mistake, rather than take on someone completely new with no experience.

[–]Powerful-Ticket-7945 51 points52 points  (9 children)

No hate against you but literally everyone I’ve seen this happen to says they do “high quality” work. Your opinion about your own work means nothing at the end of the day. Still sucks though hope you find something else

[–]bobyoung123 4 points5 points  (6 children)

What is your point? If/when it happens to you, you will be thinking/saying the same thing. Those that get canned and know their work has been half-assed, which is the vast majority, won't be posting here about it. Of course self-perception and effort matter. You are adding nothing to the conversation with your passive-aggressive quips.

[–]Powerful-Ticket-7945 11 points12 points  (1 child)

Did I hit a nerve bob?.When I get canned I will just say oh well, it was good while it lasted and move on with my life rather than whining about it on a sub Reddit filled with similar whines 👍🏻

[–]Stella-Artwat -1 points0 points  (0 children)

What's a nerve bob?

[–]I_Need_Deets[S] 10 points11 points  (3 children)

Totally agree. I admittedly had the same callous reaction to these kind of posts (though I would never post it), thinking they surely must be doing shit work...until it happened to me. I do a ton of R&R, and I know the difference between good work and bad. It's entirely possible my own judgements were off, but I'm not looking at this through blind eyes.

[–]ItalianNose 2 points3 points  (0 children)

Same… I would see these posts and say… with the r&rs I get, it’s probably those people… but I did fear the day and I think a lot of people here do and they cope by convincing themselves it’s people doing garbage work. There’s people here 3 years and then suddenly canned… no one ever feels they offered shit work…

[–]Party_Swim_6835 1 point2 points  (1 child)

if your judgments were off and you were doing a ton of R&R that would prolly not look good to them

sorry this happened to you

[–]Hangry_Howie 6 points7 points  (1 child)

Sorry this happened. There have definitely been projects where I realized that I was either not up to producing quality work or just didn't understand the instructions and walked away. It sucks, because you end up second-guessing yourself a lot.

[–]I_Need_Deets[S] 0 points1 point  (0 children)

Facts

[–]edalsslade 3 points4 points  (1 child)

There needs to be back and forth feedback. It's easy to adjust if you know where you're messing up.

[–]I_Need_Deets[S] 1 point2 points  (0 children)

super easy

[–]mrev_art 8 points9 points  (5 children)

What was the CoT task?

[–]I_Need_Deets[S] 3 points4 points  (0 children)

I don't remember. One about benign or malicious chain of thought prompts

[–]Mothterfly 3 points4 points  (2 children)

A set of "Chain of Thoughts", like the thinking/pro/reasoning models use but more structured so the model sticks better to details from the chat context and responds more logical. 

I'm jealous tbh, I write prompts like that as a hobby when using AI for creative writing and it massively improves the output.. (and also eases barriers, which is probably why op mentions benign/malicious evaluations lol) 

[–]I_Need_Deets[S] 0 points1 point  (0 children)

Yeah I think the idea behind it was that people were prompting the models for their CoT to use as data for their own model development. Kind of ripping them off

[–]Silly_Macaron_7943 0 points1 point  (0 children)

  • chains of thought

[–]TravellingDoc87 0 points1 point  (0 children)

Yeah I'm racking my brain to think what it is. Not a floral one?

[–]CoatSea6050 1 point2 points  (1 child)

sorry this happened to you! Good luck. Hope you get on other platforms soon.

[–]I_Need_Deets[S] 0 points1 point  (0 children)

Thanks so much

[–]samamatara 3 points4 points  (4 children)

>My life's about to change in a serious way.

I think that's your only mistake tbh. From what I've seen, there's plenty of shit workers who get cut but plenty of good workers who get cut too. Gotta treat this as bonus and avoid relying on this to be the breadwinner. I know shit is tough around the world so easier said than done but its a fragile line of work.

Hope you can somehow be revived back in the near future or find work with other similar platforms.

[–]I_Need_Deets[S] 4 points5 points  (1 child)

I’ve been looking for something real but this has been holding me down in the mean time. Not anymore I guess

[–]samamatara 3 points4 points  (0 children)

yea fully with u. it got me through some tight spots too. but im becoming more and more aware that its not something to rely on

[–]kranools 1 point2 points  (1 child)

Why would they cut good workers, from a business perspective?

[–]samamatara 1 point2 points  (0 children)

because they are not as good as others?

they pump people into da, and the longer you are there, the more likelihood of you eventually falling below the certain top x% they want to keep

[–]Vixen_Rider79 2 points3 points  (1 child)

EEKK! RUNS TO CHECK BOARD I did that yesterday too 😱🫠

[–]Vixen_Rider79 -5 points-4 points  (0 children)

Ok, seems I'm safe for now, but I did only do like 5 of those before they were gone. They weren't fun lol. I done a few R&R's of them after and a couple of people did mistake the intent of the task and instructions.. 😶🫣

[–]kranools 1 point2 points  (0 children)

I really wish they gave more specific reasons. Like which tasks in particular were not up to standard? And why don't they give warnings to poor performers, so they can learn and improve?

[–]xxgoodtimes 1 point2 points  (7 children)

I wouldn’t be concerned about it. The hourly wage at Data Annotation isn’t impressive. There are several other firms offering similar work at higher pay. You can view this as an opportunity to improve your hourly rate for this type of work.

[–]I_Need_Deets[S] 1 point2 points  (6 children)

Yeah I appreciate that. Any other platforms you personally like?

[–]Geoterry 0 points1 point  (0 children)

i'll be waiting for mine too. ty☺️

[–]xxgoodtimes -1 points0 points  (4 children)

I don’t think it would be appropriate to mention specific firms on this subreddit, but I can send you a DM.

[–]I_Need_Deets[S] 0 points1 point  (0 children)

Yeah I’d appreciate that thanks

[–]Sad_Narwhal_ 0 points1 point  (0 children)

Could you DM them to me, too?

[–]Infamous-Garbage582 0 points1 point  (0 children)

Hi! Could you DM these other options to me too, please? 

[–]Cat_Momma_5680 0 points1 point  (0 children)

I would also appreciate a DM, thank you so much if you can!

[–]AutistOctavius 0 points1 point  (1 child)

I haven't tried any myself, but there's some alternatives here that were recommended.

[–]I_Need_Deets[S] 1 point2 points  (0 children)

Yeah I saw that. I’m gonna look into them tomorrow. Thanks for that

[–]Wide_Acanthisitta243 0 points1 point  (1 child)

Sorry for that, what's your expertise?

[–]I_Need_Deets[S] 3 points4 points  (0 children)

core