Thursday, September 16, 2010

With Amazon Mechanical Turk, Avoid Rejection - At All Costs




Rejection in real life, be it for a job or asking for a date, can be painful, but rejection in the world of Amazon Mechanical Turk (Mturk) can have some serious effects to your earning potential. That is if you’re serious about working for Mturk for the long haul and trying to make some money online.

If you working on Mturk for any amount of time you’re likely to encounter a HIT that makes you think twice. It could be badly worded such as including contradictory instructions or be poorly worded. This should be your red flags to steer clear.

You see, your earning potential on Amazon Mechanical Turk is tied to your Acceptance rate. It’s really better correlates to your Rejection rate. To be able to access the greatest amount of HITS on Mturk, you need to have a high acceptance rate. If you drop below 95%, you will see a dramatic decrease in the amount of HITS available to you.

It turns out Requesters do have standards after all. They don’t want just anyone working on their HITs.

My example of this is a sad one because it happened to me. Early in my Mturjk days, I found a quick and easy series of HITs and while they didn’t pay a lot, I could plow through them at a speedy rate. I read the instructions and while I felt there were some contradictions in them, I proceeded anyway.

Big mistake.

The HIT did have contradictory instructions. I was categorizing some-such thing and they wanted the worker to select a categorization in an “either/or” fashion, but they didn’t design the question criteria in the correct way. It was sort of like asking a question about a flower and saying it is either red or blue --- or green or white, but it can’t be both, but you have to provide answers to both. So, with each subsequent question, I was building a slew of contradictory responses...and rejections.

My acceptance rate dropped below 95% and I saw a precipitous drop in the number of HITs available to me once I dropped below that 95% line. It takes doing a lot of HITs over a lot of time to move that needle back up above 95 depending on how far you’ve dropped down. It’s sort of like tanking your first term in college with a C, 2 Ds and an F. It’s going to take moving a mountain for a long time to get get that old GPA back up into a range where your parents will send you beer money again. If ever.

My advice is that you carefully read all instruction for a HIT. If you come across a red flag, read them again, this time very carefully.

I’ve taken a very conservative approach to low-pay, mass-issued HITS. I only do a couple and then submit them. Then I wait to see if they are accepted or rejected.

If I lose out on some HITs, I guess it’s worth the risk avoidance. That's just my humble opinion, though, but enter iffy HITs with caution.

1 comment:

  1. Yeah, I just had that happen. I had some 5 cents per hit thing and did a lot of them, some kind of "determine whether there's inappropriate content in these sentences" sort of thing and my acceptance rate just went down from 100% to 77% and is still dropping. That's as bad as getting bad seller reviews on eBay with everyone suddenly claiming their package never arrived and demanding a refund.

    ReplyDelete