Research Projects

Usher: Improving Data Quality with Dynamic Forms

Kuang Chen, Maneesh Agrawala, Joseph M. Hellerstein and Tapan Parikh¹

Yahoo! Technology for Emerging Regions Fellowship

Governments, companies and individuals routinely make decisions based on inaccurate or incomplete data stored in supposedly authoritative databases. Data entry presents the first opportunity to address data quality. However, the design of data entry forms is currently an ad hoc practice consisting of mapping all the desired information to a set of entry widgets (text fields, combo boxes, etc.), guided by heuristics and the designer's intuition. We take a more nuanced view of data entry with the aim to minimize entry errors while maintaining an upper bound on the effort involved in filling the form. To achieve this, we assess the cost and benefit of existing data entry modalities and design new ones. Using statistical algorithms trained on prior data and the human factors involved in data entry, we optimize form design in terms of field ordering, question formulation, auto-completion, and widget choice. We also introduce a collaborative and iterative form design process based on asynchronous feedback through annotations and other user-driven mechanisms.

[1]

"Usher: Improving Data Quality With Dynamic Forms". Kuang Chen, Harr Chen, Neil Conway, Joseph M. Hellerstein and Tapan Parikh. IEEE ICDE, 2010.

¹School of Information, UC Berkeley