Mechanical Turk (Mturk) is a Web service where users, turkers, are paid small rewards (few cents) for short computational task called HITs (Human Intelligence Tasks). A contractor generates the HITs, post them on Mturk and later download all the result.
TurKit is a Java/JavaScript API (developed by the User Interface Design Group at MIT) for running iterative tasks on Mechanical Turk. As of today, TurKit represents the first example of iterative tasks framework for Mturk, as it allows users to perform incremental tasks by automatically generating HITs based on the results of previous HITs.
Many applications can benefit from this iterative paradigm: turkers can take turns improving a passage of text, verify each other’s work by voting on it or implement the comparison function of an iterative sorting algorithm. In the context of SeCo, turkers can be employed, for instance, to evaluate the quality of a query response.
