Distributed estimation via paid crowd work

Consider a distributed estimation problem to be carried out by paid crowdworkers, where results are to be returned quickly and accurately. Estimation accuracy is a function of the number of workers completing the job and of the quality of the workers, both of which may be influenced by the payment offered. With limited budget, payment allocation should consider both effects to obtain best results. Since people are not deterministic, payment offers will lead to a random number of variable-quality workers, as governed by choice models. We consider average performance and focus on estimating a parameter from measurements through uniform noise. Since we have shown the optimality of the midrange estimator in specific settings of the general problem, we focus on the best linear unbiased estimator based on order statistics (BLUE-OS) under the mean-squared error (MSE) criterion. Best payment allocations are determined for single crowd platforms, joint population models and separated platform models. Illustrative numerical examples are provided.