Opti-Speech-VMT: Implementation and Evaluation

Hiranya G. Kumar; Anthony R. Lawn; B. Prabhakaran; William F. Katz

doi:10.1007/978-3-030-95593-9_19

Opti-Speech-VMT: Implementation and Evaluation

Hiranya G. Kumar
, Anthony R. Lawn
, B. Prabhakaran
, William F. Katz

College of Nanotechnology, Science & Engineering (CNSE), Dean's Office

University at Albany

University of Texas at Dallas
Topaz Labs

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

We describe Opti-Speech-VMT, a prototype tongue tracking system that uses electromagnetic articulography to permit visual feedback during oral movements.Opti-Speech-VMT is specialized for visuomotor tracking (VMT) experiments in which participants follow an oscillating virtual target in the oral cavity using a tongue sensor. The algorithms for linear, curved, and custom trajectories are outlined, and new functionality is briefly presented. Because latency can potentially affect accuracy in VMT tasks, we examined system latency at both the API and total framework levels. Using a video camera, we compared the movement of a sensor (placed on an experimenter’s finger) against an oscillating target displayed on a computer monitor. The average total latency was 87.3 ms, with 69.8 ms attributable to the API, and 17.4 ms to Opti-Speech-VMT. These results indicate minimal reduction in performance due to Opti-Speech-VMT, and suggest the importance of the EMA hardware and signal processing optimizations used.

Original language	English
Title of host publication	Body Area Networks. Smart IoT and Big Data for Intelligent Health Management - 16th EAI International Conference, BODYNETS 2021, Proceedings
Editors	Masood Ur Rehman, Ahmed Zoha
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	233-246
Number of pages	14
ISBN (Print)	9783030955922
DOIs	https://doi.org/10.1007/978-3-030-95593-9_19
State	Published - 2022
Event	16th EAI International Conference on Body Area Networks, BODYNETS 2021 - Virtual, Online Duration: Dec 25 2021 → Dec 26 2021

Publication series

Name	Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST
Volume	420 LNICST

Conference

Conference	16th EAI International Conference on Body Area Networks, BODYNETS 2021
City	Virtual, Online
Period	12/25/21 → 12/26/21

Keywords

3D model
Avatar
Electromagnetic articulography
Latency
Speech
Tongue
Visual feedback

Access to Document

10.1007/978-3-030-95593-9_19

Cite this

Kumar, H. G., Lawn, A. R., Prabhakaran, B., & Katz, W. F. (2022). Opti-Speech-VMT: Implementation and Evaluation. In M. Ur Rehman, & A. Zoha (Eds.), Body Area Networks. Smart IoT and Big Data for Intelligent Health Management - 16th EAI International Conference, BODYNETS 2021, Proceedings (pp. 233-246). (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST; Vol. 420 LNICST). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-95593-9_19

Kumar, Hiranya G. ; Lawn, Anthony R. ; Prabhakaran, B. et al. / Opti-Speech-VMT : Implementation and Evaluation. Body Area Networks. Smart IoT and Big Data for Intelligent Health Management - 16th EAI International Conference, BODYNETS 2021, Proceedings. editor / Masood Ur Rehman ; Ahmed Zoha. Springer Science and Business Media Deutschland GmbH, 2022. pp. 233-246 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST).

@inproceedings{9e3929158ede48f5a034c753c7ebe61b,

title = "Opti-Speech-VMT: Implementation and Evaluation",

abstract = "We describe Opti-Speech-VMT, a prototype tongue tracking system that uses electromagnetic articulography to permit visual feedback during oral movements.Opti-Speech-VMT is specialized for visuomotor tracking (VMT) experiments in which participants follow an oscillating virtual target in the oral cavity using a tongue sensor. The algorithms for linear, curved, and custom trajectories are outlined, and new functionality is briefly presented. Because latency can potentially affect accuracy in VMT tasks, we examined system latency at both the API and total framework levels. Using a video camera, we compared the movement of a sensor (placed on an experimenter{\textquoteright}s finger) against an oscillating target displayed on a computer monitor. The average total latency was 87.3 ms, with 69.8 ms attributable to the API, and 17.4 ms to Opti-Speech-VMT. These results indicate minimal reduction in performance due to Opti-Speech-VMT, and suggest the importance of the EMA hardware and signal processing optimizations used.",

keywords = "3D model, Avatar, Electromagnetic articulography, Latency, Speech, Tongue, Visual feedback",

author = "Kumar, \{Hiranya G.\} and Lawn, \{Anthony R.\} and B. Prabhakaran and Katz, \{William F.\}",

note = "Publisher Copyright: {\textcopyright} 2022, ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.; 16th EAI International Conference on Body Area Networks, BODYNETS 2021 ; Conference date: 25-12-2021 Through 26-12-2021",

year = "2022",

doi = "10.1007/978-3-030-95593-9\_19",

language = "English",

isbn = "9783030955922",

series = "Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "233--246",

editor = "\{Ur Rehman\}, Masood and Ahmed Zoha",

booktitle = "Body Area Networks. Smart IoT and Big Data for Intelligent Health Management - 16th EAI International Conference, BODYNETS 2021, Proceedings",

}

Kumar, HG, Lawn, AR, Prabhakaran, B & Katz, WF 2022, Opti-Speech-VMT: Implementation and Evaluation. in M Ur Rehman & A Zoha (eds), Body Area Networks. Smart IoT and Big Data for Intelligent Health Management - 16th EAI International Conference, BODYNETS 2021, Proceedings. Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, vol. 420 LNICST, Springer Science and Business Media Deutschland GmbH, pp. 233-246, 16th EAI International Conference on Body Area Networks, BODYNETS 2021, Virtual, Online, 12/25/21. https://doi.org/10.1007/978-3-030-95593-9_19

Opti-Speech-VMT: Implementation and Evaluation. / Kumar, Hiranya G.; Lawn, Anthony R.; Prabhakaran, B. et al.
Body Area Networks. Smart IoT and Big Data for Intelligent Health Management - 16th EAI International Conference, BODYNETS 2021, Proceedings. ed. / Masood Ur Rehman; Ahmed Zoha. Springer Science and Business Media Deutschland GmbH, 2022. p. 233-246 (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST; Vol. 420 LNICST).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Opti-Speech-VMT

T2 - 16th EAI International Conference on Body Area Networks, BODYNETS 2021

AU - Kumar, Hiranya G.

AU - Lawn, Anthony R.

AU - Prabhakaran, B.

AU - Katz, William F.

PY - 2022

Y1 - 2022

N2 - We describe Opti-Speech-VMT, a prototype tongue tracking system that uses electromagnetic articulography to permit visual feedback during oral movements.Opti-Speech-VMT is specialized for visuomotor tracking (VMT) experiments in which participants follow an oscillating virtual target in the oral cavity using a tongue sensor. The algorithms for linear, curved, and custom trajectories are outlined, and new functionality is briefly presented. Because latency can potentially affect accuracy in VMT tasks, we examined system latency at both the API and total framework levels. Using a video camera, we compared the movement of a sensor (placed on an experimenter’s finger) against an oscillating target displayed on a computer monitor. The average total latency was 87.3 ms, with 69.8 ms attributable to the API, and 17.4 ms to Opti-Speech-VMT. These results indicate minimal reduction in performance due to Opti-Speech-VMT, and suggest the importance of the EMA hardware and signal processing optimizations used.

AB - We describe Opti-Speech-VMT, a prototype tongue tracking system that uses electromagnetic articulography to permit visual feedback during oral movements.Opti-Speech-VMT is specialized for visuomotor tracking (VMT) experiments in which participants follow an oscillating virtual target in the oral cavity using a tongue sensor. The algorithms for linear, curved, and custom trajectories are outlined, and new functionality is briefly presented. Because latency can potentially affect accuracy in VMT tasks, we examined system latency at both the API and total framework levels. Using a video camera, we compared the movement of a sensor (placed on an experimenter’s finger) against an oscillating target displayed on a computer monitor. The average total latency was 87.3 ms, with 69.8 ms attributable to the API, and 17.4 ms to Opti-Speech-VMT. These results indicate minimal reduction in performance due to Opti-Speech-VMT, and suggest the importance of the EMA hardware and signal processing optimizations used.

KW - 3D model

KW - Avatar

KW - Electromagnetic articulography

KW - Latency

KW - Speech

KW - Tongue

KW - Visual feedback

UR - https://www.scopus.com/pages/publications/85125238046

U2 - 10.1007/978-3-030-95593-9_19

DO - 10.1007/978-3-030-95593-9_19

M3 - Conference contribution

SN - 9783030955922

T3 - Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST

SP - 233

EP - 246

BT - Body Area Networks. Smart IoT and Big Data for Intelligent Health Management - 16th EAI International Conference, BODYNETS 2021, Proceedings

A2 - Ur Rehman, Masood

A2 - Zoha, Ahmed

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 25 December 2021 through 26 December 2021

ER -

Kumar HG, Lawn AR, Prabhakaran B, Katz WF. Opti-Speech-VMT: Implementation and Evaluation. In Ur Rehman M, Zoha A, editors, Body Area Networks. Smart IoT and Big Data for Intelligent Health Management - 16th EAI International Conference, BODYNETS 2021, Proceedings. Springer Science and Business Media Deutschland GmbH. 2022. p. 233-246. (Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST). doi: 10.1007/978-3-030-95593-9_19

Opti-Speech-VMT: Implementation and Evaluation

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this