Last year, genealogy service MyHeritage went viral after introducing a new “deepfake” feature that allowed users to animate the faces of loved ones in still photos. TikTok users posted videos reacting to the technology, dubbed “Deep Nostalgia,” while bringing back relatives they had never met or those whose loss they were still mourning. To date, more than 100 million photos have been animated with the feature. Now comes the next iteration. Today, MyHeritage, along with technology partner D-ID, is expanding “Deep Nostalgia” with the launch of “LiveStory,” a feature that not only brings people in photos to life through movement, but also lets them actually speak.
MyHeritage has licensed the new technology from D-ID, a Tel Aviv-based startup working to create proprietary video reenactment technology powered by AI and deep learning techniques.
Available to developers through APIs, D-ID’s technologies have been used by a range of licensees across media, education, marketing, and more. Warner Bros. for example, worked with D-ID to allow users to personalize a movie trailer with animated photos and for a Harry Potter exhibit. Mondelēz International, advertising agency Publicis and Digitas Vietnam collaborated with D-ID on marketing efforts for a local festival. India’s short video app Josh integrated facial animation technology as a creative tool. Nonprofits and governments have also used the technology in several awareness campaigns.
With the debut of LiveStory, MyHeritage brings the latest AI technology from D-ID directly to consumers.
To use the feature, consumers can create a free MyHeritage account that allows them to try the technology multiple times for free. In addition, they must upgrade to a paid subscription for unlimited use of LiveStory.
The technology works to create an animated video story of an ancestor’s life so that they can tell their own story. This is made possible by D-ID’s patented Speaking Portrait technology, which generates a spoken video from an uploaded photo and then combines it with a synthetic voice generator. The narration of the story comes from text input by the user.
To match the lips with the words, D-ID trained a neural network on a database of videos of people speaking. The technology can work with any language, the company says. However, the MyHeritage implementation supports 31 languages, including dozens of dialects, with both male and female voice options.
“The technology is so good you don’t need driver video,” notes Gil Perry, co-founder of D-ID. image. “You just enter text and the photo and then you get the person to speak,” he says. The technology isn’t perfect, Perry admits, comparing it instead to “really good lip sync.”
After the LiveStory is created, users can view it, share it with friends or post to social media. They can also further customize the story by editing the text, choosing a different voice, or even uploading their own audio recording.
In the longer term, D-ID envisions how this technology could be used in a metaverse environment, where digital avatars of people could be animated with AI — and not just faces, but whole-body movements in 3D worlds. Perry envisions how users can upload their own childhood photos, those of relatives or even historical figures, then animate them in the metaverse and have conversations.
“Our kids will want to talk to Albert Einstein and get to know and hear him and ask him questions,” he says. “And he will answer us.” (And when universal translation is available, it can even reply in the user’s own language.)
This technology is, of course, several years down the line, but it will be built on concepts being developed today – such as Deep Nostalgia and LiveStory – when and if it will eventually happen.
Meanwhile, there’s MyHeritage and soon D-ID’s own consumer app that will help showcase its technology in a different way. The app will arrive in the coming weeks, D-ID says.
The MyHeritage LiveStory feature was announced today at the RootsTech family history conference. It works on desktop, mobile web and in the MyHeritage mobile app.
“LiveStory takes storytelling to the next level,” said Gilad Japhet, founder and CEO of MyHeritage, in a statement about the launch. “With this latest viral feature, MyHeritage continues to lead the way in the world of online family history, both in vision and innovation. Our use of AI to breathe new life into historic photos is unique, helping millions of people cultivate a newfound emotional connection with their ancestors and deceased loved ones. Genealogy is all about telling and preserving our family stories. We continue to show the world how fun and compelling genealogy can be.”
Also co-founded by Sella Blondheim and Eliran Kuta, D-ID is a team of 32 and plans to double down by hiring people in various positions in the US, UK, Singapore and Israel.
This post MyHeritage and D-ID team up to bring photos to life with both animation and speech – TechCrunch was original published at “https://techcrunch.com/2022/03/03/myheritage-and-d-id-partner-to-bring-photos-to-life-with-both-animations-and-voice/”