We present a process for validating and improving annotations made by untrained humans using a two-step machine learning algorithm. The initial validation algorithm is trained on a high quality annotated subset of the data that the untrained humans are asked to annotate. We continue by using the machine learning algorithm to predict other samples that are also annotated by the humans and test several approaches for joining the algorithmic annotations with the human annotations, with the intention of improving the performance beyond using either approach individually. We show that combining human annotations with the algorithmic predictions can improve the accuracy of the annotations.
|Title of host publication||Proceedings of ELM2019|
|Editors||Jiuwen Cao, Chi Man Vong, Yoan Miche, Amaury Lendasse|
|Publication status||Published - 12.09.2020|
|MoE publication type||A3 Book chapter|
|Name||Proceedings of ELM2019|
Forss, T., Espinosa-Leal, L., Akusok, A., Lendasse, A., & Björk, K-M. (2020). Validating Untrained Human Annotations Using Extreme Learning Machines. In J. Cao, C. M. Vong, Y. Miche, & A. Lendasse (Eds.), Proceedings of ELM2019 (pp. 89-98). (Proceedings of ELM2019; Vol. 14). Springer. https://doi.org/10.1007/978-3-030-58989-9_10