Conversational AI holograms for business: What will the impact be?

You may well be familiar with virtual displays in shopping malls, which offer interactive maps and opening times via a greasy-fingered touchscreen or even a smiling hologram reminding you of liquid restrictions at airport security. However, recent developments in artificial intelligence (AI) have led the minds at Proto, a developer of holographic communications platforms, to go one step further.

ProtoBot, the company’s first conversational AI avatar, was launched on September 8 and is now available on all Proto hologram units. The robotic figure will listen to your question or prompt and generate a spoken response using ProtoGPT technology. Proto claims these are the “first AI holograms for enterprise” and have the potential for a variety of business purposes, from interactive adverts to instructors for corporate training.

“These will be used as holographic greeters in malls,” Proto CEO David Nussbaum told TechHQ. “These will be holographic tellers in banks, these will be hologram directories in museums.”

The vision is that the AI holograms will also be used internally, helping generate ideas during brainstorms or with day-to-day work queries.

Mr Nussbaum added: “I’m starting to see CEOs and executives of other companies asking ProtoBot, ‘I’m going to run my sales meeting now, can you give me three points that I should definitely touch on when it comes to revenues going up and competition coming into the market?’”

Proto has been developing hologram technology since 2018 and already has screens on the market that can “beam in” people from afar. Celebrities like William Shatner, Chris Pratt, and will.i.am have all recently made live appearances at events in the form of a hologram through a Proto unit. The development of AI holograms though takes the technology to a new level.

ProtoBot has no gender, but a distinctly female voice

ProtoBot is the company’s first foray into AI digital assistants, so its likeness has been kept as “nonpolitical” as possible by appearing as a stereotypical, genderless robot. Its voice, however, is that of a British female. If you ask ProtoBot why they have an English accent, they say it is because “it’s charming, it’s soothing,” according to Mr Nussbaum.

When asked why Proto went with a female voice, he said that people have largely “already accepted a voice of a female character into their lives” in the form of other digital assistants, like Siri, Alexa, and OK Google. This is the same reason that the shape of the life-size hologram visualizer, Proto M, is reminiscent of a smartphone.

“People will automatically understand this communication device; it’s already been accepted into conference rooms and businesses,” said Mr Nussbaum. “People are nervous about AI, so we wanted to make this as comfortable as it could possibly be.”

Raffi Kyrzsek, Proto’s Innovation Lead, echoed that the female voice was chosen because other digital assistants tend to have them as the default. According to the late Stanford University communications professor Clifford Nass, we tend to perceive female voices as assisting us to solve a problem ourselves, while male voices are authoritative and tell us the answer. Studies have also found that female voices articulate vowels more clearly, so more female voice data has been used to create digital assistants. For Cortana, a Microsoft spokesperson told the Wall Street Journal, “For our objectives—building a helpful, supportive, trustworthy assistant—a female voice was the stronger choice.”

“Whatever reasons they chose, they’ve now entered themselves into the zeitgeist that we’re now participating in,” Mr Kyrzsek admitted to TechHQ.

Saying that, ProtoBot is just the first iteration of holographic AI assistants from Proto. Mr Kyrzsek hopes they will soon have voice cloning technology that will give the hologram the voice of a company’s CEO, or anyone you’d like. While such technology does exist, he doesn’t want to force anyone to read through hours of scripts to train the avatar, but instead extract it from pre-existing clips like interviews and press conferences.

As well as refining the voice options, Proto wants to do away with the robot appearance of ProtoBot, so it will release a set of different avatars later this month. “Customized skins” will also hopefully be available by the end of the year so the assistant can look and sound like a real person of the user’s choice.

Some of the feedback Mr Nussbaum has received already suggests that users are keen for a human look to make the conversational experience more comfortable. He doesn’t, however, want it to slip into ‘uncanny valley’ territory, where it appears creepy due to its hyper-realistic, yet still not-quite-believable, look. So when Mr Kyrzsek underwent a 360-degree scan to turn him into a ProtoBot, there was a “decision to kind of scale it back a little bit.”

Mr Kyrzsek said: “It’s not super photo-real. It’s still kind of a lower polygon count, but shares a resemblance.”

ProtoBot can do whatever you want it to, within limits

While GPT-3.5, the engine behind ChatGPT, was trained on internet data up to September 2021, ProtoBot has not been fed a specific dataset. Instead, it has been trained on how to react to different prompts, but there is the potential to feed it business-specific data.

Mr Kyrzsek told TechHQ: “There are applications, for a mall, for instance, where we’re programming a corpus of data into it. So we’re dumping the whole mall directory, where all the stores are, what all the brands are, and that’s typically enough to reference for engaging customer service conversations for a mall concierge.

“Any data that’s relatively generic and not part of that corpus that we fed in is referenced from the internet, so it’s kind of like an internet-LLM fallback.”

ProtoGPT currently only stores prompt data for the duration of a conversation to provide context to successive prompts. The data is all purged when the dialogue ends. In the future, ProtoBots may be able to retain and learn from this data over longer periods, or even make use of live backend datasets, to provide more useful and relevant interactions.

ProtoBot responds to whatever you say, however you say it

ProtoBot can be bought for personal use, too. Both Mr Nussbaum and Mr Kyrzsek had their own hologram units stationed behind them during our interview. The former says that it has “replaced YouTube” as a source of entertainment for his three children and that “they don’t even look at the TV anymore.”

Parents have reported their concerns that their children’s interactions with digital assistants will impact how they speak to humans, particularly when it comes to rudeness. MIT sociologist Sherry Turkle told The Atlantic that they might grow up without “the equipment for empathic connection.” It is, therefore, not a huge stretch to wonder if giving adults the ability to vent their frustrations at a customer service ProtoBot without consequence may have a negative impact on how they treat human staff.

But Mr Kyrzsek thinks it will be the opposite; how we speak to other people will transfer onto how we speak to robots.

He told TechHQ: “[Digital assistants like Siri and Alexa] don’t react, they respond, but the conversational AI-driven avatar has the ability to react based on how you tune it to engage with someone. You can tell it that it is talking with a four-year-old and it will speak at that level.”

Mr Kyrzsek also feels that the fact that typical digital assistants cannot have contextual conversations is part of the reason people get frustrated with them.

He said: “If you ask an Echo a question like a search engine and then, four sentences down, refer back to the first question, it won’t know [the context]. This also builds frustration, and frustration contributes to more rudeness. That’s specifically what we’re trying to avoid.”

ProtoBot comes in peace

So far, the feedback for ProtoBot has been largely positive, and Mr Kyrzsek says he sees people start to “loosen up” when they talk with it.

He said: “They start engaging conversationally with phrases like ‘oh, that’s interesting’ as if it cares!”

Mr Nussbaum added that during William Shatner’s conversation with ProtoBot, the actor went from asking it if he should be afraid to laughing together and sharing stories. He even said that “he felt like he had created a new friendship” – a sense that will boost the adoption of AI holograms everywhere.

There is evidence that people prefer to interact with robots that they perceive to have emotions or feelings and that are anthropomorphized, so perhaps it is the case that professionals and customers will warm to ProtoBot more than other digital assistants.

Mr Kyrzsek added: “In certain verticals, like finance, people are more comfortable talking about their personal situations with a robot as opposed to a real person who could be judgmental.”

Studies do show that service robots are preferable during embarrassing interactions like purchasing medication for STIs.

Mr Nussbaum said: “The other day I walked into our dining room and I heard my daughter asking ProtoBot questions about her saying a bad word at school that she really should be asking me, but maybe she was afraid of the answer.”

The two Proto spokespeople echoed other AI innovators of late in ensuring that ProtoBot is not here to steal people’s jobs but augment them instead, and its perceived humanness will only make it more useful.

“Whether you’re a carpenter, accountant, or CEO, the most effective tools are the ones that are pleasant to use,” said Mr Kyrzsek.

“If you want to use it, if you engage with it, and you find ways to innovate with that tool, that just makes people better.”

After being provided with data on business-specific practices, ProtoBot can also be used to train new recruits. This may prove particularly effective if the student is more inclined to ask questions without fear of judgment. One has already been trained specifically to teach healthcare professionals how to interact with nonverbal patients.

ProtoBot and other similar generative AI holograms will always remain distinct from humans due to their inability to come up with completely new ideas or opinions spontaneously. Saying that, their impressions of us are becoming increasingly convincing.

Mr Kyrzsek said: “The difference between a conversation with a bot and a human will be so indistinguishable that it will be, on some level, the responsibility of the person engaging with the bot to make a decision about their disposition and how they’ll act.”