Turkopticon’s Amazon Mechanical Turk Extension

As I’ve said before, making money online is a tough game. In an earlier article, I mentioned Turkopticon as a good resource for workers on Amazon Mechanical Turk (Mturk). They provide an extensive Requester reviews submitted by workers. These reviews are invaluable as you select you should work for to protect yourself from getting scammed or rejected.

Well, the Turkopticon people have taken this review approach to the next level and now have an downloadable extension that works directly with your browser while you are on the Mturk website. It’s added a pull-down menu to the AMT HITs listings that allows you to see Requester ratings from other providers. It also provides a link back to the Turkopticon database of Requesters.

This is pretty slick. So, you can now seamlessly cruise HITs on Mturk and see who to work for and who to avoid. This is a must for the discriminating Mturk provider. You’d be a fool not to use it.

Technical Note: This extension only works witgh Firefox 3.0 and above and also on Chrome.

Turkopticon: Another Resource for Evaluating HITS on Amazon Mechanical Turk

Making money online is a rough and tumble game. There are scammer and spammers. There are con artists. And then there’s Amazon Mechnial Turk (AMT) -- the online version a sweatshop, but you know what you’re getting into. Right?

Well, sometimes. Since AMT is an on-demand workplace in which workers and “employers” can come from anywhere, there’s always a chance of a con artist ready to take advantage of workers by posting jobs they never pay for.

So, how do you protect yourself? One way is by using the online forums on Turker Nation (see my recent article) and one other excellent resource is Turkopticon. Turkopticon is a basically a big online database where workers can leave evaluations and feedback on Requesters.

Workers can rate Requesters in four different categories:
Fair (Fairness)
Fast (Speed of payment)
Pay (A decent pay for a job)
Comm (Communication)

While the ratings can be quite useful, the comments left by Providers are very useful. You just have to watch for contradictory feedback. I’ve see several very good Requesters get scathing reviews from Providers undeservedly. Some Providers just fail to or don’t want to follow directions and get annoyed when they find their work rejected.

If you have any questions about a Requester, this should be first or second stop to find out if they are legitimate.

The phrase is “Caveat emptor.” In this case it’s the worker that needs to beware.

Getting Help Evaluating HITs and Requesters

Turker Nation Forums

You’re ready to make money online with Amazon Mechanical Turk (Mturk). You open up the Mturk homepage and you browse the list of HITs. After a couple minutes, you see a nice juicy HIT that provides a decent reward.

But how do you know who this Requester is and how do you know they won’t capriciously reject your work? Well, if you’ve read my earlier article, “You’re on Your Own,” then you know that Amazon certainly won’t take your side.

So, what can you do? Well, there are a number of strategies you can take to protect yourself. I have written article detailing my advice on how to Avoid rejection, but there are some helpful resources out there for you. In this article, I’m going to discuss one of the best -- the Turker Nation online forums.

Turker Nation provides an array of forum topics for the Amazon Mechanical Turk Workers...and Requesters. For evaluating a Requester, they provide a very helpful forum with the title of
Requesters Hall of Fame/Shame

The Requester’s Hall of Fame/Shame contains posts from Mturk workers where they chronicle their experience with Requesters. This thread contains a wealth of experience from Mturk Workers as they tell the story of bad Requesters and, also, good Requesters -- and this is as good as bad report because knowing you can trust a Requester is important. The forum uses positive and negative graphic icons that quickly identify who is a good or bed Requester.

If you have any suspicions about a Requester, then I would readily suggest that you check this forum to see what other Workers are saying.

Also included in this thread are lists of spammer and scammers. These are folks who use Mturk to troll for customers or personal data. Although Amazon clearly states what jobs Requester can post as a HIT, Amazon does not police HITs and I rarely see Amazon boot anyone from Mturk.

Now, backing up some. I’ve had used the Hall of Fame/Shame in the past and found contradictory reports on a Requester. One poster will slam a Requester while another poster will say that things went well with the Requester. I would imagine that along with there being some bad requester, there are some bad workers. So ultimately, you will have to discern whether or not from the posts who you can trust. Obviously a slew of bad reports on a Requester should be heeded.

Turker Nation also offers from friendly forums along with their helpful information. They have a Turker Introduction forum for new Workers to introduce themselves. They include a Questions forums for Workers and Requesters to ask find answers or ask questions.

If you’re working for Mturk, then Turker Nation should be in your bookmarks/favorites. You’d be a fool to ignore a resource like this when doing work like this.

With Amazon Mechanical Turk, You’re on Your Own

Okay, like a lot of people, you’re trying to use Amazon’s Mechanical Turk (AMT) to make some money online. You are a diligent and steadfast worker, providing your Requesters with expedient and quality work. Your Acceptance rate is a sterling 100% and the pennies are building up in your Amazon account.

Here are a couple scenarios to knock your Acceptance rate down and lower your trust not only in the Amazon Mechanical Turk system, but in humanity.

---

Scenario #1
One day you see a bunch of juicy HITs with a decent reward and decide to jump on them. You open the HITS and read the instructions. They seem somewhat ambiguous and there might be some contradictory instructions. You decide to plow ahead anyway because the HITs look easy and the reward is good. You take a couple hours and you have a lot slew of these HITs completed and are awaiting approval from the Requester.

The next day you log on to Amazon Mechanical Turn and much to your dismay you see the wreckage that was once Acceptance rate and a lack of money in your account.

Scenario #2
You see a nice HIT that handsomely rewards the worker for a writing assignment. The Requester is new the AMT game, but the HITs has a great set of instructions.

Do this to this degree.

Finish this to match this criteria.

There’s some research to be done and you hit the books (or the wikipedia) and you are armed with all your necessary facts. You then set down to write. After an hour or so, you have a piece of art ready for submission and do one final check, then hit the Submit button.

The next day you get to experience the an unqualified rejection with no explanation. You write the Requester and get no response.

---

Well, in a normal workplace, you’d go right to management in effort to fight injustice, defend inequity, and protect humanity. And you hope to get paid.

Good luck with that, I say. I have approached Amazon in the past with a complaint regarding a poorly worded series of HITs that received a slew of rejections and they cited their standard policies:

“A Requester may reject your work if they believe the answer is wrong, the HIT was not completed correctly or that the instructions were not followed. If you believe that your work was rejected in error, you may decide to contact the Requester directly.”

Then they cited their policy on their role in Amazon Mechanical Turk process:

Amazon Mechanical Turk's Role:
Amazon Mechanical Turk provides a venue for third-party Requesters and third-party Providers to enter into and complete transactions. Amazon Mechanical Turk and its Affiliates are not involved in the transactions between Requesters and Providers. As a result, we have no control over the quality, safety or legality of the Services, the ability of Providers to provide the Services to Requesters' satisfaction, or the ability of Requesters to pay for Services. We are not responsible for the actions of any Requester or Provider. We do not conduct any screening or other verification with respect to Requesters or Providers, nor do we provide any recommendations. As a Requester or a Provider, you use the Site at your own risk.”

In other words, you are on your own.

If you know this, you then can take steps to protect yourself from being taken advantage of. I have an article where I discuss how to Avoid Rejection in the dog-eat-dog world of AMT. Check it out.