Validating Untrained Human Annotations Using Extreme Learning Machines

Thomas Forss; Leonardo Espinosa-Leal; Anton Akusok; Amaury Lendasse; Kaj-Mikael Björk

doi:10.1007/978-3-030-58989-9_10

Validating Untrained Human Annotations Using Extreme Learning Machines

Thomas Forss, Leonardo Espinosa-Leal^*, Anton Akusok, Amaury Lendasse, Kaj-Mikael Björk

^*Corresponding author for this work

Finance, Helsinki

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Scientific › peer-review

Abstract

We present a process for validating and improving annotations made by untrained humans using a two-step machine learning algorithm. The initial validation algorithm is trained on a high quality annotated subset of the data that the untrained humans are asked to annotate. We continue by using the machine learning algorithm to predict other samples that are also annotated by the humans and test several approaches for joining the algorithmic annotations with the human annotations, with the intention of improving the performance beyond using either approach individually. We show that combining human annotations with the algorithmic predictions can improve the accuracy of the annotations.

Original language	English
Title of host publication	Proceedings of ELM2019
Editors	Jiuwen Cao, Chi Man Vong, Yoan Miche, Amaury Lendasse
Publisher	Springer
Publication date	12.09.2020
Pages	89-98
ISBN (Print)	978-3-030-58988-2
ISBN (Electronic)	978-3-030-58989-9
DOIs	https://doi.org/10.1007/978-3-030-58989-9_10
Publication status	Published - 12.09.2020
MoE publication type	A3 Book chapter

Publication series

Name	Proceedings of ELM2019
Volume	14
ISSN (Print)	2363-6084
ISSN (Electronic)	2363-6092

Access to Document

10.1007/978-3-030-58989-9_10

Fingerprint Dive into the research topics of 'Validating Untrained Human Annotations Using Extreme Learning Machines'. Together they form a unique fingerprint.

Cite this

Forss, T., Espinosa-Leal, L., Akusok, A., Lendasse, A., & Björk, K-M. (2020). Validating Untrained Human Annotations Using Extreme Learning Machines. In J. Cao, C. M. Vong, Y. Miche, & A. Lendasse (Eds.), Proceedings of ELM2019 (pp. 89-98). (Proceedings of ELM2019; Vol. 14). Springer. https://doi.org/10.1007/978-3-030-58989-9_10

@inbook{c7dc9fba5b43407494015241c1c9f13d,

title = "Validating Untrained Human Annotations Using Extreme Learning Machines",

abstract = "We present a process for validating and improving annotations made by untrained humans using a two-step machine learning algorithm. The initial validation algorithm is trained on a high quality annotated subset of the data that the untrained humans are asked to annotate. We continue by using the machine learning algorithm to predict other samples that are also annotated by the humans and test several approaches for joining the algorithmic annotations with the human annotations, with the intention of improving the performance beyond using either approach individually. We show that combining human annotations with the algorithmic predictions can improve the accuracy of the annotations.",

author = "Thomas Forss and Leonardo Espinosa-Leal and Anton Akusok and Amaury Lendasse and Kaj-Mikael Bj{\"o}rk",

year = "2020",

month = sep,

day = "12",

doi = "10.1007/978-3-030-58989-9_10",

language = "English",

isbn = "978-3-030-58988-2",

series = "Proceedings of ELM2019",

publisher = "Springer",

pages = "89--98",

editor = "Jiuwen Cao and Vong, {Chi Man} and Yoan Miche and Amaury Lendasse",

booktitle = "Proceedings of ELM2019",

address = "International",

}

Validating Untrained Human Annotations Using Extreme Learning Machines. / Forss, Thomas; Espinosa-Leal, Leonardo; Akusok, Anton; Lendasse, Amaury; Björk, Kaj-Mikael.

Proceedings of ELM2019. ed. / Jiuwen Cao; Chi Man Vong; Yoan Miche; Amaury Lendasse. Springer, 2020. p. 89-98 (Proceedings of ELM2019; Vol. 14).

Research output: Chapter in Book/Report/Conference proceeding › Chapter › Scientific › peer-review

TY - CHAP

T1 - Validating Untrained Human Annotations Using Extreme Learning Machines

AU - Forss, Thomas

AU - Espinosa-Leal, Leonardo

AU - Akusok, Anton

AU - Lendasse, Amaury

AU - Björk, Kaj-Mikael

PY - 2020/9/12

Y1 - 2020/9/12

N2 - We present a process for validating and improving annotations made by untrained humans using a two-step machine learning algorithm. The initial validation algorithm is trained on a high quality annotated subset of the data that the untrained humans are asked to annotate. We continue by using the machine learning algorithm to predict other samples that are also annotated by the humans and test several approaches for joining the algorithmic annotations with the human annotations, with the intention of improving the performance beyond using either approach individually. We show that combining human annotations with the algorithmic predictions can improve the accuracy of the annotations.

AB - We present a process for validating and improving annotations made by untrained humans using a two-step machine learning algorithm. The initial validation algorithm is trained on a high quality annotated subset of the data that the untrained humans are asked to annotate. We continue by using the machine learning algorithm to predict other samples that are also annotated by the humans and test several approaches for joining the algorithmic annotations with the human annotations, with the intention of improving the performance beyond using either approach individually. We show that combining human annotations with the algorithmic predictions can improve the accuracy of the annotations.

U2 - 10.1007/978-3-030-58989-9_10

DO - 10.1007/978-3-030-58989-9_10

M3 - Chapter

SN - 978-3-030-58988-2

T3 - Proceedings of ELM2019

SP - 89

EP - 98

BT - Proceedings of ELM2019

A2 - Cao, Jiuwen

A2 - Vong, Chi Man

A2 - Miche, Yoan

A2 - Lendasse, Amaury

PB - Springer

ER -