Amazon Working to Challenge Mahalo?
It appears Amazon may be integrating the human element into their A9 search results. In this video, I demonstrate that Amazon’s mechanical turk service contains thousands of assignments intended to pay turkers for their help in evaluating search results.
My take is that while Turkers can be a source of cheap labor, the tests will typically not align to genuine searcher intent, and in the world of turking that will likely affect quality significantly.
In other words, someone may take an assignment to filter results for roof+tile+adhesive, but will not in all likelihood actually care about what they find. I recognize that Mahalo may suffer from the same limitation, but in my experience, the diligence required (and achieved) from an employee working a complex task is no doubt higher than from a turker.
My assumption is that Amazon is paying less per hour than Mahalo, and may have a more scalable approach. The question is if the turkers are really weeding out less favorable items to any measurable degree.
Welcome to our community! If you like what you see, you may want to subscribe to our RSS feed!
{ 2 comments… read them below or add one }
That’s interesting stuff that we’ve obviously considered, but as you can see just checking off “good” is a very low-level editorial process.
A MTURK operator doing this task is like the person who checks to see if the printing press is printing the words in the New York Times clearly without smudges. Now, that person may be important, but so is John Markoff who wrote the story about Microsoft that they are checking for ink stains.
Again, it’s very important the John’s words be printed clearly and without smudges, but that kind of work is *MUCH* different than journalism.
Mahalo is more like journalism… the audience will tell us if the links are good or bad based on emailing us/reporting them, or by the nature of them using the service or not.
That being said, I think MTURK is a very interesting project that bring up many important issues regarding fair wages, globalization, and the value of anonymous work (which can, in fact, be very high: see Wikipedia).
I’m watching MTURK and who knows, maybe some day we will test it for something (although, I don’t understand why someone would use it if Amazon is competing with them–that makes no business sense).
best j
I absolutely agree that it makes no sense to leverage a service that a) may go away at any time and b) comes from a competitor. I think that the potential for adding value is likely to be greater than just smudge detection — if someone engineers the tasks properly. It could conceivably (unlikely) be more like DOS vs the Lisa, or Palm vs the Newton. Not as good, but a hell of a lot cheaper, and good enough.
Leave a Comment