During the past week, Jimmy Kimmel, Stephen Colbert and other celebrities have been buzzing about ‘Deep Nostalgia,’ the new Israeli-designed tool that brings faces from the past back to life by turning old photos into videos.
A demonstration of how the Deep Nostalgia tool brings old images to life. (Credit: D-ID)
Since Israeli genealogy company MyHeritage added the new feature to its app last week, more than 10 million videos have been made and shared all over social media.
Users can upload images to the photos section of the app, which is designed for researching family trees and submitting DNA samples to MyHeritage. The new feature provides an ‘animate’ option that can automatically produce a video of approximately ten seconds with the photo subject smiling and looking around. You can try it out here.
“The app has about eight different reenactment formats that were shot with different actors, and the system chooses the one that is best suited to the picture,” explained Gil Perry, co-founder and CEO of Tel Aviv-based D-ID, which created the technology. “Our technology allows us to animate the photo so it looks like a video of a person in front of us. We understood how sensitive this can be for users uploading photos of relatives who aren’t alive anymore, and we tried to make sure that the videos would create good feelings.”
However, Perry said, he actually had something different in mind when he started the company. “Our company actually started with a core mission of privacy protection. We worked with a documentary where the face of the police officer that was the main character couldn’t be shown in the film. Usually, in such a situation, the producers would blur out his face so he couldn’t be recognized, but it is hard for viewers to connect to a character like that. Our technology allowed us to swap out his face for the face of another actor that will appear in its place. The new face looks lifelike, and blinks when he blinks, smiles when he smiles, and even has the same small facial wrinkles. But his privacy and security are protected.”
“During COVID, we started to realize that our expertise of facial recognition, deep learning, computer vision, and artificial intelligence enabled us to expand into the media and entertainment market, and we believe we will be ready to lead the next disruption in that space.”
The technology D-ID used with MyHeritage is just one of its capabilities, Perry said. “We are now getting requests from museums, documentary makers, greeting card companies, photo apps, social networks and video production houses. We also have a solution that converts text or audio into a video of a talking person. Imagine that I send you a text message or a voice message, and you’d be able to watch a video of my face speaking the words to you. This is technology that wasn’t possible before. There are several things we have to get right for this to work, including voice cloning, human body reenactment, and the scene in which it all happens.”
The company sells its services to customers through an API (application programming interface) as a SaaS (software as a service) platform that can be deployed on their servers or ours, Perry said.
“Because we are the world leaders in this field that can change the media and entertainment industry, we believe that we have the responsibility to establish the principles of ethics of how this technology should be used. We choose our customers very carefully, due to the dangers we see of what ‘bad actors’ can do if they misuse our technology. Investors once told us they were concerned that our technology would be sold to terrorists. We responded that our main concern was to protect people’s privacy, but that we understood that everything can be used for good or for bad, and we want to try to guide the technology to a good place.”
D-ID was founded in 2017, and has raised more than $18 million in funding. MyHeritage, which is used by more than 62 million users worldwide, was acquired two weeks ago by US investment firm Francisco Partners for about $600m.