Microsoft Research
Microsoft Research Year 2014 Peer-reviewed
Programming Languages · Static Analysis

Optimizing Human Computation to Save Time and Money

Benjamin Livshits George Kastrinis
2014
Publication year
Microsoft Research
Venue
Peer-reviewed
Type

Problem

Abstract Crowd-sourcing is increasingly being used for providing an- swers to online polls and surveys. However, existing systems, while taking care of the mechanics of attracting crowd work- ers, poll building, and payment, generally provide little by way of cost-management (e.g. working with a tight budget), time-management (e.g. obtaining results as quickly as pos- sible), and controlling the margin of error (e.g.

Approach

working on a sample population which is largely different from the gen- eral census statistics). The problems above create significant pain points for those wanting to run large-scale surveys, such as people doing polling for political campaigns, marketing professionals, and the like. Our work unlocks the possibility of large-scale polling on a budget though the use of novel optimization strategies. Our work, is based on InterPoll, a platform for programming crowdsourced polls.

Results

In this paper, we present three static and three runtime optimizations for InterPoll polls repre- sented as LINQ queries. The former share some similarities for traditional compiler optimizations, while the latter bor- row insight from databases and real-life polling strategies. These optimizations lead to significant improvements in practice. In our experiments we observed tenfold savings in survey cost and time savings of as much as 20 hours for some of the queries.

Cite this paper — BibTeX
@TechReport{interpoll-optimizations-tr,
  title = "Optimizing Human Computation to Save Time and Money",
  author = "Benjamin Livshits and George Kastrinis",
  year = "2014",
  month = nov,
  institution = "Microsoft Research",
  number = "MSR-TR-2014-145",
}
Copied