(light music)
- Imagine we're living in a world
where we're not bounded by physical laws.
We can talk to anyone we wanted.
We can even bring back people from the dead
if we wanted to,.
There are no limits.
If you think about the movie, "The Matrix,"
the person lives in a completely virtual world.
And the society that we're progressing toward
goes more and more in that direction.
(light music)
In the future, we're going to create digitized versions
of ourselves, but what is the real future
we really want to build?
How can the technology benefit society?
It has to be a future that is healthy and it's good for us.
It should also improve us.
My name is Hao Li, I'm a computer scientist.
I am developing the future of AI-driven virtual humans
and the closest thing to a human-like thing
that's virtual is an avatar.
So the question is, why do we need virtual avatars?
First of all, we're already surrounded by virtual avatars.
Video games, for example.
Every character in a video game is a virtual avatar.
If you think about movies, it started off
with "Terminator 2" and "Jurassic Park."
After that, you had like movies like "Avatar,"
where they started to incorporate massive motion capture
This technology, it's an expansion of what's possible.
One of the visions that we have is really
to create digital humans that are indistinguishable
from real ones.
And in order to create a human that looks very real,
you need to get everything right.
The second goal is also we're trying to make a creation
of digital humans accessible to everyone.
And those are sort of like the two core challenges
that we have to solve.
In order to create a digital human,
it's a process that takes really, really long.
It's not unusual
that it would take months to create a high quality avatar.
You know, visual effects is known to be very expensive.
(light music)
This is what we call volumetric capture.
The way it works is that you can see here we have a lot
of cameras around the volume that you wanna capture.
So with all these cameras combined,
allows you basically to get a dynamic, 3D performance
of a person.
- [Cameraman] All right, Hao, action.
- In the future, we're all gonna exist in this universe.
And we're all gonna become 3D avatars
and have the ability to teleport anywhere we want.
(light music)
This kind of facility is actually ideal
for high-end production where you can scan an entire person
using multiple cameras.
But if people ever wanted to create their own avatars,
it's unlikely that people would actually
have a capture setting like this at home.
So we started to think, would it be possible
to do volumetric capture without the need
of having multiple cameras?
We want to build a technology that can generate an avatar
from a single photo.
(upbeat music)
And we actually found a way to generate an entire human body
from a single photograph.
This is how we interact with each other today.
We're using video chats, we're using our smartphones,
and most of these devices only have a single video camera.
So we have Ruilong Li here today.
He's my PhD student.
- On a screen, you can see the front side of the texture
which is actually captured by this web camera,
but you can also see the back side of the texture
which cannot actually be seen by this web camera.
- If I only see the front of myself, how do I know the back?
To get the back of a person,
you have to almost imagine what it is, right?
But what we wanted to do is see,
could we train an artificial intelligence system
to do that itself?
It's possible to train a deep, neural network
to process 3D information from a single viewpoint.
So this project, we call it volumetric teleportation
because it's not just there to capture
a prerecorded 3D video where it takes hours to process,
but it's something that you can stream instantly
from one location to another in real time.
So I could in theory, have someone
on the other side of this planet wear an AR device
and basically see myself in 3D.
The pandemic has really influenced the way we interact
with each other.
The only remote communication we have right now
is a video chat and it's just 2D.
You don't really feel the presence
of the other person in your room or really close to you.
What if people wear headsets
and can see each other in 3D remotely?
That's one of the technologies that I think
are becoming increasingly important,
especially in the days that we're physically distanced.
But in order to get there,
it's very clear that without an avatar, it's not possible.
- I'm from China.
So, my family living in China
we are doing the video conference weekly,
but sometimes I feel that's just not enough.
I kind of want to see them standing in front of me.
And I also know
that they want to see me standing in front of them.
That's another reason that I would like
to contribute myself to this field.
- We're trying to recreate a more and more realistic avatar.
We're trying to go away from, you know, emojis.
We're trying to create something that is more natural to us,
but overall, I think we have a first result.
I think it's still far from anything that's perfect,
but that's where we're at.
The human body, the human face
is something very complicated.
How light reflects on someone's skin,
all the details that are in your face, everything matters.
If something is off, it wouldn't look like a real human.
The performance of a face,
it can really tell a story, right?
And the avatars have to look as if they're real human.
Otherwise there's something really missing.
It's actually a very interesting phenomenon called
the uncanny valley effect.
So, what it means is that when people are trying
to create a virtual human or human that's synthetic,
the more you're trying to make it look real,
the harder it is to actually get there.
You have the uncanny valley,
which is if it's not entirely perfect, it feels unrealistic.
It feels in some ways, repelling.
When you try to create a computer graphics generated human,
it becomes harder and harder
the more you're trying to get real.
Because of that, one of the holy grails in computer graphics
was to create the perfect digital human.
In the first couple of years,
when I was working on this product,
it seemed like an impossible problem.
One of the challenges and also the beauty of trying
to create something that's entirely new
is really that there's a lot of unknowns.
Not every technology can have a huge success
within a few years, right?
So, certain things take a long time.
You just need to have a really long-term vision.
As a young child, I was always really fascinated by art.
As a kid one of the first thing is like,
well, given a paper in just a pencil,
can I just create a portrait of someone?
And in the beginning I would fail
and you'll feel like why doesn't it look like the person
and what are the things that are missing?
So, I was really fascinated by how do painters have
the ability to recreate something that almost looks
like a photograph?
And in some ways, what we're doing is still very similar.
We're trying to create digital humans.
And if there's one thing that's wrong about it,
we immediately notice that it doesn't look real.
What we are trying to do is we're trying
to create something that other people think is impossible.
And even to a point where we think it's impossible.
(light music)
To make sure that we're creating avatars
that look more and real,
there are several things that we're doing in parallel.
First thing is it has to look good to us.
If it doesn't even match our standard,
then there's something wrong.
So our standard is basically,
it has to look at least as good as a triple-A game.
So, what we're gonna see here
is our automatic avatar digitization technology.
That all you need to do is take a picture of yourself.
(camera clicking) And from that single picture
it can generate an entire avatar.
(light music)
So here you can see that, although it's high quality,
there's some difference with a real human.
- The believability of an avatar
really depends on the eyes and mouth.
People really focus on those.
So, it's really important to spend a lot
of time around those areas.
So, that can involve like animation
which is really important
and sometimes an underrated aspect.
- It's really bad animation.
People are gonna get put off.
So in order to make things feel lifelike,
they have to feel like they're listening to you.
They have to feel
like they're understanding what you're saying.
I've worked with companies before
and normally it takes a really long time
to generate these characters,
but it's nice here.
They get generated so quick
because of the tech that the geniuses like Hao are making.
- We work with digital artists in a very different way
than normal visual effects or game studios.
The content that they create is actually used
as training for our system.
So, the algorithm in some ways is learning how
to create an avatar from the artist.
But what we're trying to do is really think
about how can we create new type of capabilities?
What you see here obviously is
that the avatar is actually moving according
to my facial expressions.
You have a face tracker that is tracking my face
in real time.
You can actually animate the characters.
You know, I could play a video game right now
and have myself in it, or my friends in it.
We are actually interacting with machines on a daily basis.
We talk to our phones, we talk to our Alexa and Siri.
We need to obtain information.
Now, the use case of an avatar is to basically
have the most natural interaction
which is just to talk to another person.
And we want to go beyond just creating a virtual assistant
that can respond to some prescripted questions.
We want to create an avatar that has the ability
to have its own mind.
That has the ability to
have natural conversations with someone.
This is our fully autonomous, virtual human.
It's a really personal project for me.
It's actually a digital version of my wife.
We digitized her look, her facial expressions,
and also created a little AI behind it.
So I created this avatar because,
first of all, she liked the idea
and people are very sensitive to their own faces.
So, I think creating a avatar of my wife
is the perfect test, right?
She could see this doesn't look like me at all
or this is acceptable.
Hey, do you like running the company with me?
- I do, but not as much as I would like.
I like to spend time with my dogs.
- Oh, that's a great answer.
- We work at the intersection of art and science and math
and blending between them intelligently is a big part
of what makes the avatar look extremely realistic.
- The path for us is very clear that in a year or so,
I think we're gonna get like 90% accurate
to a real human.
One of the things that we learned very early on
is that you do have to have a responsible approach
to developing these type
of algorithms simply because they're extremely powerful.
- This is a fully digital version of me.
- There's definitely a need for having a good understanding
of what the technological capability is
and what harm it could potentially have.
So, a deepfake is a technology that allows you
to swap someone's face with yours.
You could make that person say
and do things that person has never done.
So around 2017, I saw in the news
some technology called deepfake.
The first use was to use it for non-consensual pornography.
And that was the first time I heard about deepfakes.
When I saw that, I was like this is disturbingly similar
to what we have been developing for years.
And for us, the technology was used
to create more realistic avatars
and make them accessible to anyone.
And suddenly you see that my technology
has a potential danger if it's used in the wrong way.
And it was an obvious threat
to our research community as well.
For anyone who's building a sustainable future,
we're responsible for being able
to build safeguards into the technology.
What is important for us is to first educate the public
that these kind of technologies are possible.
So, we modified our technology to showcase
how fast deepfakes are advancing.
So, he can actually move around.
You can smile and you can see how it's possible
to generate very convincing faces and manipulate them
in real time.
With a little bit of production effort,
deepfakes are going to become perfectly indistinguishable
from a real footage.
(speaking foreign language)
We actually got a lot of requests
from agencies and organizations,
including DARPA, on where it's going, right?
So, we started to work together
on an approach that allows us to not only detect
if something has been manipulated,
but also can we find who manipulated it?
Does it come from a certain source and also looking
at what are next generation threats for data manipulation?
(light music)
- Come here, come here. (dog barking)
Let's go, let's go.
Come here.
- It's really hard to predict what can happen, right?
But I'm also so very optimistic about humans
because we've always been able to adapt to new technologies.
It feels really amazing, right?
You know, sometimes to get away from computers
for awhile and really to reconnect with
what is actually natural to humans.
(speaking foreign language)
We're kind of like part of building what the future is.
So, we constantly have to think about like
what it potentially could look like.
But, I'm not sure if I wanna live
in a entirely digital world.
I think I would be fine with living
in a world where you have both physical and digital,
but a completely digital world,
I think, we as humans are not made for it.
We will live in a world where things become a hybrid.
Really like augmented reality.
Augmented reality is in some ways already part
of our daily lives.
When we create content that we put on social media
we add digital content to it,
whether it's 2D or 3D.
But how can the technology benefit society
and go beyond just customer service or online shopping,
but really reinvent what the next generation
of human machine interaction is?
What if we have a way to create
a fully autonomous virtual teacher?
So, you could basically create a virtual avatar
that is teaching math to kids.
And the avatar has the ability to get feedback,
create a personalized teaching for every individual.
I think that's what our future looks like.
So, we can already talk to anyone anytime we want,
but in the future we can actually bring someone
in this place here.
And for certain things,
you really want the closeness of another person.
(speaking foreign language)
In the end, it's really about us.
It's about interconnection.
We're reinventing a world
that's important to us and we feel comfortable in.
(light music)