By Will Marks, General Manager of MING Labs
‘Bringing Emotion Into AI’ was published in the Summer 2021 edition of the German Chamber Ticker. Editor: Noga Feige, Senior Editor of Ticker Magazine. Visuals: Matter Design
Voice user interfaces went from zero to one in less than a decade. Now, digital assistants are commonplace, and brands seek to differentiate and connect with customers on an emotional level. This article covers the latest state of this technology and outlines future trends and implications.
2021 is the year where the number of installed digital assistants is forecast to outnumber humans (Ovum, 2017). Siri’s launch a decade ago has spearheaded their widespread adoption, with all the major tech companies now shipping equivalent services. But with most of them acting as voice-only ‘servants’ – an opportunity existed to build more engaging, brand-specific characters. China has recently seen the emergence of this new generation of Virtual Personal Assistants (VPAs) – who combine a visual avatar with playful personalities to enable a new era of emotive AI interactions.
Spotting this opportunity to improve user & brand experience, Ford became the first global auto brand to launch a VPA character, debuting it in this year’s Shanghai Auto. MING Labs has been helping them with the strategy and design of this program, and is happy to share its understanding of this rapidly emerging space.
The History of Virtual Personal Assistants
The most (in)famous forerunner to today’s VPAs was Microsoft’s Clippy, the widely mocked MS Office assistant. Researchers at Stanford University (Reeves & Nass, 1996) discovered that human brains treated computers emotionally similar to people. The gender and appearance of software could influence how people responded to it. By anthropomorphizing a simple suggestive algorithm, Microsoft bet that they could capitalize on these findings to realize the future of a more intuitive User Interface (UI) while achieving commercial gains through a brand mascot. Ultimately Clippy failed, panned by users as an annoying distraction and ranked by Time magazine as one of the 50 worst inventions of all time.
A stunted experience due to technology limitations ensured the failure of this first generation of VPAs. However, improvements in AI Machine Learning applications like Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU), opened the door for a new generation of voice-driven digital assistants led most prominently by Siri. We have seen the likes of Baidu, Alibaba & iFlytek launch a suite of voice assistant services within China. Baidu’s DuerOS alone handled six billion interactions in Q4 2020. The wide usage of smartphones and, more recently, smart speakers have provided the platform for this second generation of digital assistants to succeed where Clippy and its peers failed.
The New Generation of VPAs
Despite its failure, the example of Clippy was instructive in that its strategic objectives remain sound. Improving the customer experience of a service through a more natural, personality-driven character is a valuable opportunity, especially given the IP commercial potential of developing a brand mascot.
Led by NIO, Chinese Electric Vehicle (EV) brands were the first to successfully realize this. NIO identified the VPA sweet spot that cars presented with the high adoption of hands-free voice interactions, powerful screens & chipsets, and the increasing convergence of tech and auto across both positioning and product.
NIO launched Nomi in 2019. Nomi is part Siri-style digital assistant, part playmate, located within a dedicated head style screen atop the dashboard. In an echo of the doomed Clippy, Nomi uses accessories to demonstrate personality across different scenarios. With an emoji-style abstraction of a face, it is able to simply convey emotional tonality. This design is influenced by a phenomenon known as ‘Kindchenschema’ - using baby-like biological triggers, such as large heads, big eyes and round cheeks, to be perceived as cute and likable.
In the market for a few years already, it is easy to forget how much of a radical departure it was. Automotive has historically been a serious, mature space. Male-centric and frequently using feature-driven positioning. The addition of a cute, childlike digital helper was a risky product decision that paid off, helping to quickly build a strong emotional bond between the brand and its customers.
While Nomi was the first mover, other local EV startups such as Xiaopeng, LiXiang, Bestune (一汽奔腾) & Enovate (天际汽车) were quick to follow. Xiaopeng’s VPA took the anthropomorphized character a step further, launching in its P7 sedan a character (小P) which featured a full body including arms and legs.
Through research conducted at MING Labs we found that Nomi tested best amongst local users. The simplicity of Nomi’s avatar meant that it had a more universal appeal, with more detailed visual representations performing poorly among some people. Avatars that featured arms and legs could be perceived as incongruous, given these characters are operating in the pure digital space.
Many of these first follower executions have their limitations. Easter egg interactions such as the VPA acting ticklish upon touch feel gimmicky and lack credibility – not unlike Clippy. Most of the avatars have no obvious connection to the brand and have struggled to differentiate themselves. Nomi stands out in large part due to the industrial design of it’s ‘head’ – the ability to turn and make eye contact is a powerful emotional tool that resonated well with consumers.
Next Steps for Virtual Personal Assistants
With VPA’s becoming the norm for new car launches in China, thoughts turn to what next.
We can expect VPAs to continue to expand their role outside the car. Mobile apps, car showrooms, and even smart homes are set to be explored as brands use their VPAs as intelligent brand ambassadors, building a consistent narrative across the customer journey. NIO has already started to use Nomi as a tool in their efforts to build a broader lifestyle brand positioning, debuting a broad range of merchandise in Shanghai Auto.
Inside the car, their importance is also set to increase. By 2030, 68% of China’s passenger-kilometers will be powered by autonomous vehicles (McKinsey, 2020). Critical to the operation of these level 5 autonomous cars will be a “Virtual Chauffeur – cognitive AI that fulfills all occupants’ explicit and unstated needs.” VPA characters are set to be the face of this next generation of predictive AI, the primary interaction point within a car as auto brands move away from the core experience of driving to a broader mobility play that features socializing, entertaining, or working.
Considerations When Building a VPA
If your business identifies a strong use case for building a VPA there are several things to consider.
1) Character. Within automotive, the same VPA could notify you on hazardous tire pressure, as well as play a game with your children. The tonality needs to be fitting across scenarios in order to build trust and positively enhance the overall brand experience. For any design may commence, a clear north star on the personality of your VPA has to be considered and validated. Is it your best friend or your pet? Is it a literal representation of the device itself or rather a sentient being that lives inside it? For Ford, MING Labs aligned on utilizing the personality of a side-kick, believing that for Ford, the driver is the hero and in any daily narrative, the VPA is there to support them in their missions.
2) Technology. Few companies have the capital and expertise to build and maintain the technology stack required to deliver a Voice AI assistant. More likely, you will partner with an existing product such as those offered by iFlytek, Baidu or Ali. Careful consideration needs to be given to their suitability. Does the provided software development kit enable you to deliver your roadmap? How collaborative can they be for requirements not accommodated within the current product? No visual layer can compensate for a fundamentally weak experience, so this may be the most critical decision you take. For international companies, global voice AI services are unlikely to be fit for purpose within China – Ford, for example, partnered with Baidu to localize its voice AI services within their Sync2.0 platform.
3) Voice. Most current VPAs are using stock voices provided by the original technology provider. However, with humans perceiving 38% of personality delivered by a voice itself, there is a missed opportunity to optimize the VPA x Brand fit. More classic voice devices such as GPS head units have long utilized celebrity voices to provide added value to users, and VPAs need similar attention in order to deliver the required quality of brand experience.
4) Brand Fit. A key litmus test for a successful VPA would be: if you took it out of the interface, would users readily identify it as visually & emotionally fitting your brand? If the answer is no to both of these, then the VPA can be considered a failure. The VPA is one of a constellation of brand touchpoints, and given its highly engaging, highly interactive form, if it fails to cohesively fit into the overall brand experience it can do more harm than good. Ford resolved this by utilizing the iconic logo’s oval shape while ensuring that the design featured visual & motion effects that represented the company’s China positioning.
5) User Experience. A core challenge that most auto VPAs in China face is a weak user experience (UX). Frequently added as an afterthought to the interface, they are more commonly a distraction. A core premise of a good UX is being both credible and desirable, but most VPAs never leave the screen – even when the user is focused on safety critical tasks such as driving. For Ford, MING Labs opted against having a distracting avatar and used a responsive scaling up experience. When driving, the VPA is a simple oval, subtly placed in the header. Once activated, it emerges in an overlay to directly engage with the user.
In Conclusion
Humanity has become accustomed to the continual emergence of powerful new technologies with the ability to transform the way we live and work. But for companies seeking to employ these in customer-facing applications, emotional authenticity is critical to success. The addition of a personality layer to interactive touchpoints has the power to improve usability, drive engagement, and build a unique piece of branded IP. Given the proof of concept that we have seen in China EVs, we can expect them to spread to other verticals such as SmartHome and Robotics, and it will be interesting to see if brands move to build them across East Asia and beyond. Regardless, we are looking at a future where technology and personality are set to combine to realize a new utility in intuitive digital assistance.
Will Marks is the General Manager of MING Labs in China – a global innovation and digital product agency. MING Labs combines research, UX design and software development to help corporations across Automotive, Industry 4.0, IoT and Consumer Tech with their digital transformation efforts.
