MICHAEL THOMSEN: Hi, I'm Michael Thomsen,
and I am a product manager on the Dart and Flutter team.
We're here today to tell you about when
you can benefit from using concurrency
in your Flutter apps.
As a developer, you'll want to create
apps with fast and smooth user interfaces with no UI jank.
Sometimes other code can block the UI from updating.
[PHONE RINGING]
Sorry.
Sometimes other code can block the UI
from updating, triggering such UI jank.
Concurrency can help you move that code
so it runs in the background and no longer causes interruptions,
like when someone interrupts you with a phone call
in the middle of a video shoot.
MICHAEL GODERBAUER: Hi, I'm Michael Goderbauer,
and I'm an engineer on the Dart and Flutter team.
As the other Michael said, concurrency
is a great technique for running code in the background
while keeping your app responsive
and animations running smoothly.
I'll show you today how you can offload
heavy computations in your app to a background worker using
isolates in Dart.
Isolates are a construct for running multiple threads
of execution concurrently.
MICHAEL THOMSEN: Our first topic is synchronous
versus asynchronous function calls
and how this relates to the event loop of a Dart-based app.
MICHAEL GODERBAUER: All Dart code runs in an isolate.
Each isolate has a chunk of private memory
and a single thread running an event loop.
Lots of events happened during the lifetime of an app.
Events are, for example, generated
when the user taps the screen, I/O finishes loading,
or when Flutter needs to repaint to update
the UI shown on screen.
As new events happen, they are added to the event queue.
When the isolate runs, it grabs the events
from the queue one by one and processes them
on a single thread in its event loop.
For smooth rendering, the Flutter framework
adds a repaint request to the event queue 60 times
a second, one for each frame.
It's important that these are processed
by the event loop on time.
Otherwise, the app's UI will become
unresponsive and animations will stutter.
MICHAEL THOMSEN: Here's something
I worked on recently, which illustrates this issue.
This is a small app for doing image processing.
You can load an image from disk or the network,
and you can apply image effects like the sepia color filter.
Let's take a look at the build method.
First, we check if there's a current image.
And if so, we render it.
Then we have a progress indicator,
which is shown when the show progress variable is true.
This is used to provide feedback for long running operations
like loading and image processing.
Then we have a number of buttons, like the Load button.
Let's take a look at the event handler for the Load button.
This starts by showing a file picker,
and then it does the actual loading of the image.
As you can see here, then it starts
by setting the show progress variable
to true to enable the progress indicator.
Then it reads the file, and then it
unsets the show progress variable
and updates the current image.
Here's the running app.
Let's start by loading an image.
We'll select photograph and click OK.
That's odd.
We was supposed to get a progress indicator.
We clearly set the show progress variable to true in the code,
but it never appeared.
Michael, can you explain what's going on here?
MICHAEL GODERBAUER: Let's start by reviewing
what happens behind the scenes when
you load an image in this app.
Once a file has been selected, the event loop
of the main isolate is running the code of the load image
function, which as you mentioned,
sets show progress before it reads the bytes of the file,
so you might expect that the UI would paint the progress
indicator on the next repaint event.
However, the read as byte sync call in the load image function
runs synchronously and blocks the event loop until all bytes
of the image have been loaded.
This can take a few frames.
The event loop doesn't get a chance
to process the pending repaint event until after show
progress has been unset again at the end of the load image
method.
As a result, we end up with an unresponsive app that
misses a whole bunch of frames and we never
get to see the progress indicator.
If you change the file loading to await
the result of an asynchronous read call,
the loading of the file is done on a separate VM thread.
While that thread is busy, the main isolate
can continue to run and process repaint events
to update the progress indicator animation.
When the VM thread is done processing,
it adds an event to the queue of the main isolate
to signal that the file has been loaded.
The main isolate then resumes the execution
of the load function, which removes the progress
indicator again.
MICHAEL THOMSEN: Got it.
Let me see if I can make that change to my app.
We will start by reading the bytes using async.
This returns a future, so we need to await it.
Oh, and then whenever we use async,
we need to add the async keywords to our function.
Let's hard reload and try the load button again.
We will take an image, click OK.
Great, here's our progress indicator.
And it looks like here's the loaded image.
Thanks for helping me, Michael.
A second feature I've been working on for my app
is the ability to perform image processing, for example,
to apply a sepia filter to add a warm color tone.
Here's my current code.
First, I set the show progress state true,
then I do the image processing, update the current image,
and set show progress to false.
But that's odd.
When I click the sepia button, it
looks like the whole UI is frozen
and nothing seems to be happening.
There's certainly no progress.
And wait, here's the finished image.
What's happening now, Michael?
MICHAEL GODERBAUER: Your app is doing the image processing
synchronously in the main isolate again.
The apply sepia function never yields
to allow the event loop to process
other events like drawing the progress indicator.
The next time the event loop can process a repaint event
is after show progress has been set to false again.
Therefore, the progress indicator
is never actually drawn on screen.
MICHAEL THOMSEN: Based on the previous example,
maybe I should try and use async.
Let me try to wait for a future, for example,
for the duration of, say, one second and see what happens.
Oh, we need to add async keyword again.
Let's hard reload and try that instead.
We'll click the sepia button.
Here's the-- wait, we got a little bit of the progress
animation, but then the whole UI just froze again.
MICHAEL GODERBAUER: In this attempt,
the asynchronous Future.delayed API
allows Dart to wait until the specified time has
passed without blocking.
In that time, the event loop can process other events,
like repaints, and draw the progress indicator on screen.
However, once the time has passed the code
after the awaited future is scheduled to run
in the main isolate again.
While the filter is applied to the image in that block
of code, the main isolate is blocked for a few frames
and cannot service the repaint events on time to update
the progress animation.
Hence, we see a frozen animation on the screen.
Just because an API is asynchronous
doesn't mean its work is done concurrently.
It can still end up blocking the main isolate,
as we've seen here.
Whenever you have work to do that cannot be completed
between two frames, it's a good idea to offload the work
to another thread to ensure that the main isolate can produce 60
frames per second for smooth user experience.
In Dart, you can offload work to another threat
by spawning a new isolate.
This worker isolate then runs concurrently
with the main isolate without blocking it.
When it has completed its processing,
it can return the result back to the main isolate.
The easiest way to do this in Flutter
is to use Flutter's built in compute function.
This function will spawn an isolate,
pass it some data to kick off a computation,
and terminate the isolate once the computation is done.
While the computation is running in the work isolate,
the main isolate is free to process other events,
like updating the animations shown on screen
or responding to user input.
MICHAEL THOMSEN: Great.
Let me try that.
First, we'll remove the future code,
then we'll add the call to the compute function.
I believe that takes two arguments, the function to call
and then the state to pass to that function, which
would be our image.
Let's go ahead and clean this up, and we can hot reload.
Oh wait, this returns a future, so we need to await it again.
Now we can hot reload and try the sepia button.
Great, here's our progress animation,
and it appears to be running smoothly with no jank.
And we get the processed image.
Our image processing app demonstrated
how to use the compute method to move a function to a background
isolate.
That's a great technique for short-lived functions,
like processing a single image or parsing a large JSON blob.
But sometimes, you have processing
that runs continuously throughout the lifespan
of an app.
Michael, do you have an example of that?
MICHAEL GODERBAUER: Yes.
I've been working on a new game where you can play tic-tac-toe
against Dash.
However, whenever the advanced Dash AI powering the game
is scheming up its next dashing move,
the entire UI freezes and Dash stops dancing,
as you can see here.
To fix this, I need to move the game engine to its own isolate.
That frees up the main isolate to keep
the UI responsive and animations running smoothly.
Since the game engine needs to continuously keep
track of the game state, this is not suitable
for a short-lived worker isolate.
Instead, we need to keep the isolate running
for the entire duration of the game.
MICHAEL THOMSEN: In the image processing app,
we applied a filter to the image with the compute function.
This automatically handled creating a worker isolate,
passed the input, and then returned the result
and exited the isolate.
This was an example of a short-lived isolate.
For a long running isolate, we need to do the setup manually.
First, we need to create the isolate.
We can then pass messages between the main and worker
isolates using ports.
And finally, we can exit the isolate.
Michael, what does that look like in your game?
MICHAEL GODERBAUER: Here's an illustration
of the architecture of my new and improved tic-tac-toe app.
We still have a main isolate responsible for drawing
the user interface.
It spawns a long running worker isolate,
which will own the game engine.
When we need the game engine to compute the next move for Dash,
we send a message to the worker isolate.
While it is calculating the next move,
the main isolate can continue to run
so we avoid any interruptions or jank in the UI animations.
When Dash move is ready, a message
is sent back from the worker, which
we can process in the main isolate
to show that move on screen.
This continues for each move until the game ends.
Let's see what this looks like in actual code.
When the game starts, we create a new concurrent game engine
and the start method is called.
This first spawns a new isolate passing
at two parameters, the entry point for our worker isolate
and its input.
As input, we provide it with the sending endpoint of our
receive port so the worker can send messages back
to the main isolate.
We've also wrapped the received port in a queue,
so we can simply await the first message
from the worker isolate.
Typically, the first message is a send port,
which allows us to send messages back to the worker.
With the communication channels established,
we send the first message to the isolate to start the game
and await the initial UI state as response.
The other methods of the game engine
are implemented similarly.
We forward the method call by sending a message
to the isolate and await its response.
The actual computation is done on the worker.
This doesn't block the main isolate,
so the UI with all animations continues to render smoothly
at 60 frames per second.
Let's shift our focus to the code for the worker isolate.
Its entry point receives the send port
we provided to the spawn method as input.
This allows us to send messages back to the main isolate.
To receive messages from the main isolate,
we create a receive port and sent its send port back
to the main isolate.
Next, we instantiate the existing game engine
and listen for incoming messages on the receive port.
For each message, we call the corresponding game engine
method and sent its return value back
to the main isolate via the send port.
That's it.
Let's run the game with the modified engine again.
As you can see, the animations now
continue to run smoothly while our advanced
Dash AI calculates its next dashing move concurrently
on the worker isolate.
MICHAEL THOMSEN: In summary, for Flutter apps,
the main thread is responsible for rendering the user
interface.
To ensure that your UI runs smoothly without jank,
your code must produce each frame
within the frame budget at least 60 times a second,
or even more frequently, on higher end phones,
with faster refresh rates.
Any code that takes longer to run than the frame budget will
cause jank, and should be moved to another isolate.
MICHAEL GODERBAUER: In this talk,
we've shown three ways to run on another thread.
For system APIs exposed in the .core libraries,
use asynchronous calls awaiting the result.
For short-lived background tasks,
use the compute function.
This takes care of spawning the worker isolate and handles
the communication between the main and worker isolate.
Finally, for a long running background tasks,
use the isolate APIs to spawn a worker isolate.
The isolate doesn't share any memory--
they are isolated.
So to communicate between them, you send messages via ports.
MICHAEL THOMSEN: You can find more information
and both app examples using the links shown under Resources.
Thanks for watching.
We hope these tips will help you create user interfaces that
run smoothly with no jank.
[MUSIC PLAYING]