Summer/Fall Publications

It’s been a busy summer!

In addition to the publications mentioned previously (The VoxWorld Platform for Multimodal Embodied Agents, Exploiting Embodied Simulation to Detect Novel Object Classes Through Interaction, and A deep dive into microphones for recording collaborative group work), members of the SIGNAL Lab were involved with a slew of other papers that were presented during the summer or will be in the fall:

Challenges and Opportunities in Annotating a Multimodal Collaborative Problem Solving Task and Multimodal Features for Group Dynamic-Aware Agents were presented by SIGNAL Lab Ph.D. student Mariah Bradford at the Workshop on Interdisciplinary Approaches to Getting AI Experts and Education Stakeholders Talking (Bridging AIEd) at AIEd in Durham, UK, in late July. These papers address questions in feature extraction and annotation in small group tasks for the NSF AI Institute for Student-AI Teaming (iSAT).

iSAT speech-based AI display for small group collaboration in classrooms won the Best Interactive Event Award at AIEd! SIGNAL Lab Ph.D. student Ibrahim Khebour worked on this demo for iSAT.

Exploring Correspondences Between Gibsonian and Telic Affordances for Object Grasping was presented at the 2nd Workshop on Annotation, Recognition and Evaluation of Actions (AREA) at ESSLLI in Galway, Ireland in early August. This paper is adapted from part of Aniket Tomar’s Master’s thesis work on grasping affordances.

Ph.D. student Sheikh Mannan’s paper Where am I and where should I go? Grounding positional and directional labels in a disoriented human balancing task applies AI and language models to a novel task involving a sophisiticated notion of embodiment. It will be presented at the Conference on (Dis)embodiment, hosted by the Centre for Linguistic Theory and Studies in Probability (CLASP) at the University of Gothenburg in Sweden on September 16.

Finally, A Generalized Method for Automated Multilingual Loanword Detection will appear as a poster at COLING 2022 in Gyeongju, Korea in October. This paper was a true team effort by researchers Abhijnan Nath, Sina Mahdipour Saravani, Ibrahim Khebour, Sheikh Mannan, and Zihui Li, in which we present a novel approach to detecting loanwords across arbitrary language pairs. Its companion paper Phonetic, Semantic, and Articulatory Features in Assamese-Bengali Cognate Detection, led by Abhijnan, will be presented at the associated Workshop on NLP for Similar Languages, Varieties, and Dialects (VarDial).

Congratulations to all these students, and many others who have work under review or in preparation!

Citations

• Bradford, M., Hansen, P., Lai, K., Brutti, R., Dickler, R., Hirshfield, L. M., Pustejovsky, J., Blanchard, N., and Krishnaswamy, N. (2022). Challenges and Opportunities in Annotating a Multimodal Collaborative Problem Solving Task. In Workshop on Interdisciplinary Approaches to Getting AI Experts and Education Stakeholders Talking (Bridging AIEd). International AIEd Society.

• Castillon, I., Venkatesha, V., VanderHoeven, H., Bradford, M., Krishnaswamy, N., and Blanchard, N. (2022). Multimodal Features for Group Dynamic-Aware Agents. In Workshop on Interdisciplinary Approaches to Getting AI Experts and Education Stakeholders Talking (Bridging AIEd). International AIEd Society.

• Dickler, R., Foltz, P. W., Krishnaswamy, N., Whitehill, J., Weatherly, J., Bodzianowski, M., Perkoff, M., Southwell, R., Pugh, S., Bush, J., Chang, M., Hirshfield, L. M., Showers, D., Ganesh, A., Li, Z., Danilyuk, E., He, X., Khebour, I., Dey, I., Puntambekar, S., and D’Mello, S. K. (2022). iSAT speech-based AI display for small group collaboration in classrooms. In Interactive event at International Conference on Artificial Intelligence in Education (AIEd). International AIEd Society.

• Tomar, A. and Krishnaswamy, N. (2022). Exploring Correspondences Between Gibsonian and Telic Affordances for Object Grasping. In Workshop on Annotation, Recognition and Evaluation of Actions (AREA). ACL.

• Mannan, S. and Krishnaswamy, N. (2022). Where am I and where should I go? Grounding positional and directional labels in a disoriented human balancing task. In Conference on (Dis)embodiment. ACL.

• Nath, A., Mahdipour Saravani, S., Khebour, I., Mannan, S., Li, Z., and Krishnaswamy, N. (2022). A Generalized Method for Automated Multilingual Loanword Detection. In International Conference on Computational Linguistics (COLING). ACL.

• Nath, A., Ghosh, R., and Krishnaswamy, N. (2022). Phonetic, Semantic, and Articulatory Features in Assamese-Bengali Cognate Detection. In Workshop on NLP for Similar Languages, Varieties, and Dialects (VarDial). ACL.