SYNTHESIA
In this project we wanted to find out if it is possible to substitute a human being as a presenter in an event so that a human being didn't need to record video of the whole presentation and it would be difficult, or even impossible, for the audience to figure out that the presenter actually doesn't exist. The event we were targeting was the Match XR 2020 event that was held in AltspaceVR in November 2020.
After looking around for different possibilities we decided to create our presenter so that the presenter's talking head would be created digitally and our model person would then record his' speech for presentation and speech would be placed on the lips of our artificial presenter, thus, allowing it to present on behalf of the real person. We chose to use Synthesia's service for our purpose since it proved to be straightforward and easy to use.
Here is what we did:
- Our model person Jari Lahti, Head of Innovations at Yle was recorded in front of a green screen where he read out five pieces of specific sample text each resulting in one minute of video and sound.
- The five sample videos were sent to Synthesia and they created a digital twin of Jari, "an actor", in their service.
- Next we recorded the speech we wanted to present during the presentation.
- Speech was uploaded to Synthesia's service and we also created background slides for the presentation and positioned our actor to the right side of the video to be created.
- Next, Synthesia's service composed the actor, speech and background slides into a video file that we were able to download and use further.
- Actually, we created the presentation in a few individual parts in order to allow us to work more in parallel during the creation process.
- We also recorded some "effect sounds" like coughing and placed it in the middle of the presentation in order to make our artificial presenter look more human..
- Final video of the presentation, where an artificial person presents on behalf of our model person, was uploaded to YouTube and played back to the audience by event organisers.
What did we learn:
- It can be done and it is actually fairly easy
- One needs to decide beforehand whether a synthetic voice or voice of a real person fits one's use case
- Use of teleprompter while recording training material allows artificial person to look directly at its audience
- In the long run, artificial presenter does not look that natural and one should figure out different tricks to make it look more lively and natural
- This technology is not real-time, yet. It takes roughly half an hour to place words or speech on an actor's lips.
The final presentation can be seen on YouTube:
