Prof Vincent Wade, director, Adapt. Image: Science Foundation Ireland
Sam Cox finds out how the Adapt research centre puts the human at the centre of future digital technologies.
The Adapt research centre for AI-driven digital content technology is led by Trinity College Dublin but spread across eight third-level institutions in total. It is one of five Science Foundation Ireland research centres to receive next-stage funding in 2021.
Part of Adapt researchers’ work is to show how they are achieving impact. To convey data and stories to the Government and the public that demonstrate they are ‘doing the right thing’ with public investment.
Adapt’s metrics to date are impressive. The number of researchers the centre supports is doubling, from 199 up to almost 400. For every €1 publicly invested in Adapt, more than €5 is leveraged back to the Irish economy. And its combined research publications exceed 1,700 articles.
All of this represents advancement in the digital technology industry and illustrates Ireland’s increasingly significant global position in the area of AI and digital content technology.
‘Adapt’s research spans an arc from AI and content technologies to the fundamental principles for society’
– PROF VINCENT WADE
In its first phase, Adapt’s research focused on the digital technologies of the future. As these technologies become more and more seamless in our lives, we are examining how they are integrated into our world. But what does successful integration look like?
Adapt’s research will address these challenges under three areas, and the additional funding affords the team the opportunity to augment and expand its research expertise and talent reach across Europe.
Above all, human empowerment will be at the core of what their success means. Talking to centre director Prof Vincent Wade, he said Adapt is about making technology that “empowers us and does not make us feel slaves to the digital”. This might sound dramatic, but at the heart of their ethic is keeping the human in focus. As such, the first strand of Adapt’s research strategy focuses on digitally enhanced engagement.
Sustainable data use
Speaking about the evolution of Adapt into its second phase of funding, Wade stressed that their research is more interactive, and so much more fluid. “It is how to empower individuals to use this technology. How society can live in a digitally successful way. How to be protected in privacy and control, and to have the right governance in place to do this correctly. Adapt’s research spans this arc, from AI and content technologies to the fundamental principles for society.”
Wade emphasised the move towards systems only knowing what they need to know at the time they need to know it, and nothing else. He highlights that past perspectives often relied on collecting as much data as possible in order to personalise systems, but this no longer has to be the case.
Once the right information for the situation is in a program, superfluous data collection isn’t necessary and, in the interest of personal data rights, shouldn’t be collected. Increasing use of synthetic or generative data reduces privacy issues even further. Reducing the carbon footprint of the models is another benefit of this approach, as less data needs to be stored and processed. Sustainable data use and privacy-preserving models is the future, marking a move away from service providers who may be in the market of selling data.
Wade’s own area of research involves personalisation and adapting these technologies to the user. His three main metrics of success in this area are effectiveness, efficiency and user satisfaction. Effectiveness means that if you were trying to learn something online, at the end you feel you can solve the problem. Efficiency asks whether you were able to learn that concept faster than if you had just taken the general course. And user satisfaction is all about feeling supported and more confident. Did you feel in control? Did you feel that was helpful?
Poor personalisation in the past has been about prior experience. Wade explains that if the system knew you liked steak, then it would recommend more meat dinners. The more you like something, the more the system recommends it. To be satisfying, it is essential to know how humans operate. Designing systems that understand what a person is looking for, and which can integrate seamlessly into devices to empower the individual, is a central aim.
Machine translation
Language, then, is a key area for Adapt. Researchers here seek to capture, communicate and translate language in a way that feels genuine for its users.
With language at the heart of most human interaction – and as more and more of the world fits into your pocket – machine translation is closing the gap in an increasingly global economy. Adapt is at the forefront of bridging these cultural gaps.
The centre’s second strand of research looks at digital content transformation, and enabling content to flow freely across different languages, cultures and modalities via machine translation. Deputy director Prof Andy Way leads this area. He talks about the progress within his field and the billions of words translated every day by Google. Translation on this scale involves the balance of speed and accuracy to function effectively in the real world.
‘Adapt set up eight machine translation systems to support the sharing of all information that was known about Covid-19 in its early days’
This was no clearer than in Adapt’s rapid response to Covid-19. As with many researchers across the world, they contributed to helping where possible during the pandemic. Adapt set up eight machine translation systems to cross the language barrier and support the sharing of all information that was known about the virus in its early days.
Machine translation technologies often see only incremental improvements in accuracy and speed, with the effect of these changes perhaps not even registering with users. What does a 1pc increase in accuracy over the system six months ago actually mean on an individual basis? The priority, Way explained, isn’t necessarily how much better the translations are, but how much better users feel when seeing them. This is especially the case in the work of professional translators.
The user interface, the accessibility, the manner in which it empowers its translators – all of these are the metrics that are essential to the design. Up to now, empirical evidence measured fewer keystrokes, fewer edits required, and the reduction in the length of time spent editing, but these fail to assess the cognitive load on human translators. If a document took 45 minutes to translate rather than an hour, how did the translator perceive that time frame, and what did their interactions with the technology entail?
Augmenting the human experience to help translators do their work more quickly, rather than replacing workers with artificial intelligence trained on huge data sets, allows people to increase their free time. You can do the same work but more efficiently.
Real communication in a virtual world
If you think the complexity of spoken word can confuse a system, try introducing tone and body language. This is the work of Prof Naomi Harte who is heading up the digital content transformation strand at Adapt.
Speaking of Zoom video interaction, Harte said: “I’m interested in turn-taking. You speak, I speak, you speak. Human conversation doesn’t work like that – we overlap all the time. You send me little signals: ‘You can keep talking because I’m listening’. And then you send signals that I might need to stop because you want to take a go. And then I’ve worked out what I’m going to say before you’re finished.”
Speech, Harte pointed out, is multimodal. Think back to any conversation in a loud pub. While you’re looking at your friend speak, you can understand what they’re saying. If someone gets in the way, they become unintelligible. Often, unbeknownst to you, you have started to lip-read and integrate that information into your understanding. Head nods, facial expressions and hand gestures all feed into our communication processes.
‘I can read psychology literature but that’s no replacement for someone with that deep understanding that complements what I do’
– PROF NAOMI HARTE
Now, more than ever, many of us may have become acutely aware of this through our collective Zoom fatigue. Without our usual multimodal cues, it has become increasingly exhausting to communicate over the web.
“It’s the online equivalent of when you’re on a footpath and you don’t know whether to go left or right. Natural communication is challenging online. It’s great in one way – you still see people and engage with them – but there are a lot of complexities there,” said Harte.
Understanding these complexities and integrating their solutions into our technologies requires a multidisciplinary approach. “That’s one of the beauties of Adapt,” said Harte. “I can interact with people who have stronger knowledge in that area. I’m the engineer, at the end of the day, and I can read psychology literature but that’s no replacement for someone with that deep understanding that complements what I do. That’s the power of a centre like this.”
The answer to these complexities isn’t just more data. While many companies may boast about 98pc accuracy, or the ability to have better systems than their competitors, Harte’s work entails looking at the last 2pc. Those who are forgotten by the mainstream consumer.
“We look at niche groups, such as ageing speakers. Children’s speech is also very challenging. As a human, I don’t need to turn a knob in my brain to understand a strong accent, or someone who uses grammar incorrectly. Bad grammar turns a speech recognition system on its knees, whereas humans can still understand,” Harte explained.
Her team’s work in Adapt going forward is about addressing language with the least amount of data. This allows the technology to be applied across the board: to languages that are dying out, languages where use varies from its dictionary definition, or any communication with fewer speakers. Adaptive signal processing is central to Harte’s research and involves understanding the underlying commonalities of a signal and adapting to recognise them. Success isn’t just empowerment for the majority but considering the often-overlooked and making sure no one is left behind.
AI and society
When designing technologies for all of society, the bigger questions inherently branch into ethics. Within the centre, PhDs have included expertise in areas as diverse as religion, business, law and sociology. This is to be the focus of the third research strand.
Prof Dave Lewis says that these questions were present in the earlier iterations of Adapt but today, as we prepare for the societal transformations of a digitally mediated world post-Covid-19, this has become an increasing focus. And it’s not just because of obligations such as GDPR. It is the pressing need to anticipate the future implications of AI on society and related data governance obligations that merits this increase. No one currently has the answer.
What are the rules that organisations should have and should be conforming to as AI is rolled out with increasing frequency? The debate happens at multiple levels, and important questions have to be asked. Can businesses be trusted to implement these solutions, or do they need to happen at a governmental level? Are international bodies such as the European Union the solution instead?
Just as there are international recommendations concerning human rights or environmental practices, Lewis thinks that privacy, ethics and governance need to be considered at a global level. Some of these solutions are technical. One example is a consent receipt: a web cookie that contains the details of what you have signed (or signed away) when you hurriedly click ‘I agree’ on a webpage, and provides some user protection.
Other examples are more fundamental. Issues such as trustworthy AI and AI terminology: “Does everyone mean the same thing when they say AI? We’re now at the stage where we’re getting noticed at an international level and getting ratified. We’re providing and contributing to these technical standards, and we’re convening a lot of these [international] working groups,” said Lewis.
For Lewis and Adapt, it’s important to capture the range of the experiences that are affecting all levels of society. At the academic level, discussions and debates around best practice, where advice is also shared, are commonplace. But he also stressed that science that engages the public is crucial to a centre like Adapt.
Adapt has a Citizens’ Think-Ins programme designed to engage with the public and different communities to find out what they think. This has been made considerably more difficult since Covid-19 but capturing both sides of the coin is essential, as is ongoing engagement: being both transparent and accountable. It’s not enough to ask groups what they think, Lewis highlights. It is essential to feed this information back into the research.
With smart technology now ubiquitous, empowerment must be on the user’s terms. In delivering this, Adapt’s vision is for a technological world that is transparent, user-friendly, and ethical.
By Sam Cox
Sam Cox was named the science and technology winner in the 2020 National Student Media Awards (Smedias). This award category is sponsored by Science Foundation Ireland and includes a €1,000 bursary to support and encourage up-and-coming science and technology journalism.
The 2021 Smedias are now open for entries. The deadline for applications is 15 April 2021.
