Pros
You work from home and it is a legit job.
Cons
You only get paid for 4 productive hours (minimum, they let you work 5 hours per day maximum at certain peak season), this is while you are connected into their system rating pages, one after the other, just like a telemarketing job where the calls don't stop. Is the same system. You are measured both by the speed (you have to rate more than 100 pages per hour) and by the efficiency (your margin of mistakes is very low). During your work as annotator or judge, you are constantly screened (and stressed). You have 2 types of audits, one is internal the other one is from their customer. You never know when are you going to receive an audit. If the audit is wrong, which can often happen since some of the guidelines for rating are a bit confusing because they depend mostly on personal opinions, they do not "discount" that audit against you from your global performance. Besides, you are not paid for looking at your email, reading newsletters, reviewing the question board (where everybody is allowed to ask questions - thanks God, because you have no real nor personal supervisor), attending to weekly forums, posting questions or posting tech issues (all of these tasks are done very often and might be very time consuming - it represented to me 1 extra hours per day and I was only allowed to invoice 5 minutes per day). Every 2 weeks they release a report and if you don't meet their mysterious metrics (these are changing continuously), you are removed from production. This means 1 week (or two depending on their tech issues with their system) without salary. They might let you go through a new qualification exam, but you don't get paid for that and it takes you at least 1 or 2 hours of preparation + 1 hour for the exam. I advise you that in can be a very stressful and frustrating job since although it is legit (you do receive a check) it is very frustrating and unfair. Besides, I have a friend that was hired by them to post positive reviews in several forums, so don't trust the 100% satisfactory reviews you read here. I cannot say they are not real, I just say they might not be real.