A researcher from Athlone Institute of Technology’s Software Research Institute is exploring how immersive technologies, such as augmented and virtual reality (AR/VR), can be used to help stroke victims find words.
Conor Keighrey, a PhD candidate, is applying these technologies to healthcare with the goal of helping people with aphasia – a type of language impairment brought on by stroke – to recover their speech.
Testing semantic memory
Using a game development engine, the Ballinasloe native has created a multimedia speech and language assessment capable of testing semantic memory – a type of long-term memory that stores general information about the world.
The speech and language assessment, which is ordinarily paper-based, manifests as a virtual speech and language therapist who examines users’ ability to understand image and word pairings.
Conor Keighrey, PhD candidate, Athlone Institute of Technology
Delivered through AR and VR headsets and a handheld tablet device, users are shown a sequence of images, such as an eye, and asked to match it to its bedfellow – in this case a pair of glasses.
When users select the correct answer, they hear positive affirmations from the virtual speech and language therapist. With VR, the user is fully immersed in the experience; with AR, information is superimposed on a real world environment.
Measuring physiological responses
Curious to understand users’ experiences of these immersive technologies more fully, Keighrey decided to measure their physiological responses during the virtual speech and language therapy.
“As part of the experimental methodology, I captured and reported physiological measures of stress, such as electrodermal activity and heart rate, and application interaction events to gain significant insight into user performance,” he says.
His findings were recently published in IEEE Transactions on Multimedia – the world’s leading journal for multimedia research – and suggest that AR and tablet users benefit most from the experience.
“These results demonstrate how embracing new and emerging technologies can unlock a variety of untapped metrics which can deliver a more holistic overview of assessment performance to a speech and language pathologist,” he explains.
Transformative potential of immersive technologies
According to Keighrey, the transformative potential of immersive technologies in how we acquire skills and experience and explore hard-to-grasp concepts, is starting to be recognised.
“The importance of these technologies has never been greater. Students across the world are utilising e-learning platforms and video conferencing rooms to attend lectures due to the current COVID-19 pandemic,” he says.
“In the future we may be able to use VR to transport students to a virtual classroom environment where they can interact with visual content as they learn about science and history.”
Using AR headsets
He is quick to note that the power of AR should not be underestimated either: “Some universities are using AR headsets to assist in the training of future nurses and doctors. These devices can be used to provide an interactive and immersive experience of human anatomy alongside lectures.”
“Conor’s research has contributed fundamental knowledge in user perceptual quality of multimedia experiences,” says supervisor Dr Niall Murray, who is a leading researcher in the field of immersive technologies and a Science Foundation Ireland funded investigator in the ADAPT and CONFIRM centres.
“Getting a paper published in the top ranked multimedia journal in the world is a huge achievement and a testament to the quality of research ongoing in Athlone Institute of Technology. The peer review process for IEEE Transactions on Multimedia is incredibly stringent.
“This is a significant milestone for Conor as he enters the final stages of his PhD. I look forward to supporting him in the next steps of his career and would like to congratulate him on this fantastic achievement on behalf of his co-supervisor Dr Ronan Flynn and myself.”
Conor Keighrey is an Irish Research Council Government of Ireland postgraduate scholarship recipient and is expected to achieve his PhD in autumn 2020.
An early access version of his paper, entitled “A Physiology-based QoE Comparison of Interactive Augmented Reality, Virtual Reality and Tablet-based Applications” is available on IEEE Xplore and the AIT Research Repository (Open Access).