Google I/O 2011: Bringing C and C++ Games to Android

Galpin: Hello, and welcome
to Bringing C and C++ Games to Android.
I'm Dan Galpin. I'm a developer advocate
on the Android team working with game developers.
So about this talk--
um, we're gonna expect
that you guys are C and C++ developers.
If you don't know C and C++, you probably shouldn't be here.
Hopefully, you've gotten some Android background,
because we're not gonna cover all the basics here.
This is considered a higher-level talk.
Um, game development-- the development experience
is awesome.
We are not gonna teach you
how to become a game developer here, either.
And, however, we are going to expect
that you may or may not have ever touched the NDK
or, in that case, JNI.
So our agenda for today is going to be programming Android
and C and C++,
using the NDK, bringing your game to Android,
troubleshooting tips and tricks, and tongue twisters
and best practices.
So to start off,
um, Programming Android in C and C++.
Um, and I'm gonna be joined, by the way,
by Ian Ni-Lewis, who's also a developer advocate here
on the Android team doing game development--
uh, helping game-- helping out game developers.
So to start off with,
the NDK provides support
for C and C++ development.
NDK is the Native Development Kit,
and the idea is people write code
in other languages than Java.
[inhales sharply] I know. It's kind of shocking.
And sometimes-- sometimes this code works,
and they want to reuse it,
so we--so we've made the NDK partially because
there's tons and tons of great code out there,
whether it be open-source code,
whether it be code that you've developed,
that you're gonna want to reuse.
Also, as it turns out,
Java is not the best language for everything.
There are times
when you're doing things that are really math-sensitive.
There are times in which you are doing things
that are seriously hard-core
where you're gonna want to accelerate things,
and you're gonna want to take advantage of things like VFP.
You're gonna want to take advantage of things like NEON,
okay, for--for supercool stuff.
And of course, I have heard that there are other platforms
that people write games for other than Android.
And--I know. It's shocking. [laughter]
Galpin: Um, but you might want to port those entire games over,
and you want to--a method of doing that
that allows you to get high performance
and consistent quality.
So let--let me let Ian take it away from here,
and he's gonna talk about what's in the box.
Ni-Lewis: I haven't been this embarrassed
since I, uh, started teaching class at DigiPen
and thought that it started a week after it actually did.
I found out that all my students wanted to kill me.
Uh, I apologize for being late, but yes,
um, as Dan was saying, the, uh, the NDK
is a way to program C and C++ in Android.
How many of you have used the NDK?
Okay, awesome.
The rest of you, you know, bless your lucky stars.
Um... Galpin: [laughs]
Ni-Lewis: The NDK was actually kind of difficult to use
in the past.
It's gotten a lot better this year,
and we're actually pretty excited
about the changes that we've made.
Here's what the NDK is.
It starts out with a toolchain,
and if you're familiar with, uh, Linux programming
or Mac programming, this sort of toolchain
is not gonna seem at all out of place.
It's basically GCC and--and Make and Build
and all these nice things that you're used to.
It also has a build system.
Uh, you're not actually required to use this
as a--as an NDKr5.
And, at first glance, it seems a little overengineered,
but it's based on the same platform build system
that we use for Android.
And it makes it really, really easy
to write modular code
and--and build up an application of multiple shared libraries
in C++.
And I'm told that,
if you try and do the whole toolchain from scratch yourself,
it's considerably more difficult.
So you might want to at least start with the made files
that we've got for you.
Um, and there's also a set of headers and libraries,
so libraries that interface with the underlying system.
Now this is a really important point.
In fact, it's actually the most important piece of the NDK,
and you might be asking yourself, "Why?"
In fact, if you're like me, you're probably saying,
"Well, wait a second, I heard Android's Linux,
"and Linux has always been C++,
so why do I even need this stuff," right?
Well, it turns out that,
although Android is based on Linux,
Android processes
have a very important difference,
um, and that's the--you know, we've always had the Dalvik VM,
and every Android process loads the Dalvik VM.
Application code is expected to run in the VM,
over the top of this set of frameworks,
um, that is the Android SDK, in Java.
Now what the NDK is trying to do
is let you get some code down here
under--underneath that layer,
uh, where only C and C++ code
have--have been written before.
And the problem with this
is that, for a long time, the Android operating system
has been on, uh, able to rely on this framework layer
for backwards and forward compatibility.
So the problem with just sticking some code down there,
which you know, you know, you can do--
you're not dumb,
you can--you can compile C code on Linux--
but if you do it without the NDK,
you're gonna have a real problem here,
because the set of system libraries that we talk to,
uh, that--that implement the Android system
in C and C++ have never been stabilized.
There's never been a reason for them to be stable.
We change them every time we revise the operating system.
We're continually making them better
and switching things around
based on the needs of the underlying OS.
So what we needed with the NDK more than anything else,
uh, was to create
a stable ABI,
and with that stable ABI,
then we could finally get the framework classes
out of the way and let your application code
live in both Java land or Dalvik land
and C and C++ land.
With first revision of the NDK,
we--we launched some basic libraries.
Basically, the things
that we thought were gonna be most interesting
to high-performance, uh, programmers--
you know, obviously, the basic stuff
like--like libcs, math libraries, uh, and OpenGL.
Now with Froyo, we also introduced a library
called libjnigraphics
which let you talk directly to the 2-D rendering layers,
uh, in a bitmap layer, as well as the 3-D OpenGL stuff,
and with Gingerbread,
we actually made a concerted effort
to make the NDK,
and--and the Android system in general,
more attractive to game programmers,
so we introduced some new input models.
Um, we used libEGL,
which is a--the standard way
of getting OpenGL contacts.
Um, we introduced OpenSL,
so finally, you didn't have to go back to Java
just to do sound.
And we introduced a library called libandroid,
which allows you to access some of the SDK framework
from C++.
Uh, most notably, it's now possible, in Gingerbread,
to write, uh, an activity
completely in C++,
and it's, uh, it's also possible,
in Gingerbread
to access the AssetManager from C++,
which, if you ever tried to access the AssetManager
from--from C in previous versions of the NDK,
it was incredibly difficult.
So we're excited to see those changes.
Well, now that we've talked a little about what the NDK is,
we should probably make it really clear what it's not.
This is not the end of Dalvik.
Dalvik is still a great way to program Android,
and it will always be the--the premier way
of programming Android.
You always have, uh, SDKs written in Dalvik first.
But...
it does give you a way to--to write C--
C and C++ code if you've got an, uh, an investment in that.
It's not always gonna be higher performance.
Dalvik is a fast virtual machine,
and not everything is gonna benefit from NEON instructions
or from hand-tuned assembly code.
So it's not always gonna be the right choice for every game.
Is it the right choice for your game?
That's gonna depend on a lot of things.
Keep this in mind--
Dalvik is the premier development environment
for Android.
It's a productive development environment.
It has garbage collection and lots of nice libraries.
But on the C and C++ side,
uh, lots of programmers know C and C++
that don't know a Dalvik language.
Uh, it can be faster,
and it can leverage existing libraries
like open-source libraries that have been written in C and C++
and are available for free.
So it may or may not be the right choice for you.
Fortunately, you don't have to make
an all-or-nothing choice.
And we'll talk a little bit about how Java and C and C++
can coexist.
By the way, I do apologize for my voice.
I was way too enthusiastic yesterday.
Whoo! Google I/O!
Um...
Let's talk about how to use the NDK.
All right, first off, you're gonna have to install it.
This is a really simple step unless you're on Windows,
in which case, you're kind of screwed.
[laughs] No, I--so on any system,
you're--you're gonna want to make sure
you've got Java and Ant up to date.
I don't know, on my MacBook Pro,
I had to do, like, a-a MacPort to get for these things.
Um, but on Windows,
you're almost certainly gonna want to download Cygwin.
Um, that's not-- that's not a restriction
we put on regular Android programming,
but, you know, the build scripts for the--for the NDK
just have never been ported for Bash.
Um, it is actually possible
to build without Cygwin.
Um, I'll mention a couple of ways
in a bit on how to do that,
but it's highly recommended.
So then there's a sort of a nice flow that we go through.
You set up a project.
Um, you write some makefiles.
You're gonna write some interface classes
and glue them up with some JNI magic.
Um, and then you're gonna go
into sort of your build, deploy, and debug cycle
that--that characterizes any software project.
Let's drill down on these a little bit.
All right, to create the project,
there's actually several ways you can do this.
The one on the slide is the out-of-the-box way.
You run Android create project,
and that gets you, as I'm sure you're aware,
an Android project.
There's also a couple of third-party projects
that I want to mention just because they're so cool.
There's a product called vs-android
on Google code that, uh, will create an Android project
in MSBuild, so a lot of game developers
use Visual Studio.
If you like that sort of thing, um, that's a possibility.
There's also a project called Sequoyah
that's part of the Eclipse project.
If you load that as an Eclipse extension,
it will actually help you create, uh, an NDK project
using the confer to C++ project wizard
in, uh, in Eclipse.
Can you tell I'm a V-- a Visual Studio guy,
'cause I just said "wizard"?
Do they even call it that in Eclipse? I don't even know.
Um, but it's gonna create the-- either way,
you're gonna--you're gonna get a few different files created.
There's--there's an XML file that Ant uses.
There's a couple of properties files
that you usually don't need to care about.
Um, and you are also actually gonna have to create
a couple of files yourself, unless you use
that Sequoyah, uh, project that I talked about earlier.
Um, it's two makefiles.
These are actually fairly standard makefiles
in--in the sense that they used new make syntax,
but they fit into that fancy build system
that I mentioned earlier.
So they don't require nearly as much work as a normal makefile.
The first makefile is called Android.mk.
It's mandatory.
The build system expects to find this,
and if it doesn't, it will complain.
And it just contains the names
of the files that you're compiling
and any flag overrides that you want to do,
so, you know, you want to set C flags for your project,
fine, you can do it here.
There's also a somewhat optional makefile called application.mk.
Um, the idea is you might have several modules
in your application, you know, several different libraries.
Each one gets its own Android.mk,
and then the application.mk pulls it all together.
But the application makefile
also does some-- some other important things.
It's a--it's a place to set your global debug flag settings,
um, and more--more importantly than anything else,
it's a place to set up your STL.
And one of the great new features
that we've finally been able to add in NDKr5
is real STL support,
including exceptions,
uh, that you get with just this one line of code
in your makefile.
Now you need to write an interface,
because remember that, unless you happen to be targeting
the 4% of devices that run Gingerbread right now,
you're probably gonna want to start out in Java
and then call in to C++.
So you write a Java interface
that's gonna wrap your C++ in something that looks like Java,
and to do that, you use the native keyword.
This is actually really easy.
You just decide which functions are gonna be, uh,
implemented in C++,
and then you use the native keyword
to do--um, to define those functions
inside your Java file.
After that, you just need to generate stubs
in your C and C++ code
which you can do with a variety of tools.
Eclipse has a tool for this.
There's also a command line tool in the JDK called javah,
and it just generates a file that looks a little like this.
Remember that--oh, darn, these little slides.
Uh, remember we had two public native void--
or public data functions in our sample code.
If we passed the class with these functions in them
to javah, it's gonna generate code that looks like this.
Basically, you have a couple of JNI export functions
and... the, um,
the names of the functions are a concatenation
of the class path and the class name
and the function name, all separated by underscores.
And the parameters to those functions
are transformed into JNI types
that--that are in C--
you know, jint, for instance.
Uh, and then there's a couple of extra parameters.
One's a pointer to a JNI environment,
which is sort of the object
that you use to talk to the Java system from C++,
and there's also gonna be a handle to your object.
Or, if the function is static in Java,
there's gonna be a handle to your class.
We'll show you how to use that in just a second.
Now you might be thinking to yourself,
that looks really easy, and it is.
Just by giving the function the proper name,
the Java compiler and the--and the Runtime
know exactly how to find it in your c--C and C++ code.
And they find it with a name lookup
in--on a vtable that's sort of automatically generated for you.
But... there are times
that you might want to do this manually.
For instance, let's say that you've got some functions
that you know you want to implement in C,
but you don't know exactly
what the implementation is until Runtime.
A good example of this would be
if you've got some really heavy-duty math library,
and you've got an implementation that uses NEON
and then another implementation that doesn't.
Well, not all CPUs have NEON,
so you might want to delay the binding of that function
until you know what your instruction set is going to be.
If you wanted to do something like that,
it's also actually fairly easy.
You just make your own function table.
Um, it's just an array of structures
that has the name of the function.
It has this cryptic little string.
That's just telling you what the function signature is.
In this case, uh, two integers, uh, or the function
takes two integers and returns void.
If you don't want to memorize
this particular bit of gobbledygook,
you can use a, uh, tool called javap
to generate that, or, once again,
Eclipse has a menu item that does this for you.
Um...
and then finally, the last thing in that structure
is just the point of the function.
Once you've created that table,
when you're ready to register up your--your interface,
you just call a function called register natives,
and you pass in a pointer of that table that you created.
And that way, you could actually have
multiple different implementations
of the same functions inside the same library
being hooked up at Runtime to your Java code.
Of course, it's cool that you can call C++ code from Java.
It's also kind of nice to be able to go the other way.
That's not super--I'm not gonna tell you that's easy,
because it's not one line of code,
but it is pretty straightforward.
And this is where we use that class or, uh, object handle
that we talked about earlier.
Um... using object handle,
you can get a handle to its class.
Using class handle,
you can get a handle to a method,
and once you've got a handle to a method,
you can use a function on that JNI environment,
uh, one of the call method
family of functions
that differ by return type.
And--and that allows you to actually make a call
from C and C++ back into Java.
Excuse me. Mm.
So that's all the coding.
Now we get to build, and that's pretty normal.
If you've done any Java or--excuse me--
any, uh, Android programming before,
you probably know the second two steps.
It's the first step that's a little different.
You run ndk-build, which is a shell script
that just launches all those makefiles,
stitches everything together,
copies everything to the right directories,
and makes sure
that all of the C and C++ code that you've compiled
is where the packager expects to find it.
Then run Ant to package it, run Ant deploy or--
I'm sorry--Ant install to deploy it,
and we're ready to debug.
Now this is actually the most important, cool thing
that we've done in the past year with Androids.
We actually made it possible to run a real instance of GDB
and do multi-threaded debugging
on Android devices.
The NDK GDB script is a shell script
that does a whole lot of heavy lifting.
It starts by sanity checking your setup and making sure
that you can actually debug the code that you think you can,
um, and then it's gonna start your application
in an instance of GDB server.
The GDB server is this tiny little thing
that just communicates
with a real GDB across, in this case, a named pipe.
Why is it a named pipe instead of a socket?
So we don't have to ask for internet privileges to debug.
So GDB connects to this, uh, named pipe on your host.
It thinks there's a socket on the host,
because it's been, um, aliased.
And once GDB is running, you can attach to that
with any tool that supports GDB--
I know of at least three.
Eclipse supports GDB.
Um, Visual Studio supports GDB
if you buy an add-on called WinGDB.
Xcode, which I've never used, but I hear it's cool,
also, uh, it supports GDB.
So you can use these IDEs
or you can use the, uh, command line interface
to GDB, as well.
Now I'm just about to turn it back over to Dan,
and we're gonna talk about some tips and tricks,
and it's really actually the most interesting part
of the talk, but before we do,
I want to leave you with one tip of my own,
just from experience debugging.
Make sure that, when you start debugging the NDK,
you have a Gingerbread device.
No matter what you're targeting,
the debug support pre-Gingerbread
was not phenomenal.
Gingerbread changes all that, and it allows you
to actually debug multithreaded applications
and have a hope of actually figuring out
what's wrong with your code.
So remember to do that,
and then I'm gonna turn this over to Dan
for some ideas on how you can bring your games to Android.
Galpin: Thanks, Ian. [applause]
Galpin: Um, so this part of the talk
is subtitled "Saving Ian's Voice."
And, uh, but, uh, it's really good--
it's really great to have him here to talk about,
uh, a lot of the stuff that's going on in--in the NDK.
So this part of the talk, we're gonna be talking about
actually bringing your game to Android
and what that actually involves.
So let's first talk about what devices
have been accessing Android market.
This is basically the freshest data.
It's a 2-week period. We published it just this week.
And as you can see, it's really exciting.
We've had great uptake for Android 2.2.
65.9% of devices checking into market
are now running Froyo,
and 94.7%
are running Android 2.1 or higher.
So this is fantastic,
because it means that you can take advantage
of many of the latest features of Android
without having to eliminate much of the market at all.
But NativeActivity is just starting.
Right now, we're at 4.3%,
so for non-tablet-specific titles,
things that are not going to require Honeycomb,
we want to be able to take advantage of NativeActivity
when it's useful, and I'll give you an example.
Um, how many people here have heard about the Xperia PLAY?
Okay. Cool device.
Um, it has a touch pad on it, and that touch pad
is only accessible right now through NativeActivity.
So a lot of people are in a position
where they want to write an application that's compatible
that can take advantage of those touch pads,
but they still want to hit the rest of the market.
So let's talk about how to do that.
So we can do something
called platform-specific resources,
and we can define boolean values there.
And this is how that works.
We create resources in a values--
in a values-v9 directory.
Inside of there, we create two values,
one called, in this case, "atleastGingerbread"
and the other called "notGingerbread."
From there, we can just go into the manifest.
Now we enable one activity,
if atleastGingerbread is true,
and one activity if notGingerbread is true.
Voil�, We have an application
that now can run NativeActivity on Gingerbread
and will run whatever standard activity on everything else.
Now there are some things you'll want to do in your native code
to take advantage of this.
Oh, let me actually show you--
there's what I was gonna show you there.
Um, there's some things you're gonna want to do
to take advantage of this.
One is you're gonna want to create stub interfaces
to your game engine.
Obviously, you do not want to HardCode the dependencies
to NativeActivity in your game engine
and include the whole thing twice.
It's gonna make the download a lot larger.
It also means maintenance can potentially be larger.
So what we're going to do
is we're going to create some stub libraries.
We'll talk about that really fancy build system
we had before.
This is what the Android.make looks like now.
As you see, it just contains a pointer
to three other libraries we want to build.
There's the application.makefile.
It actually lists each one of those libraries.
Inside of the individual directories,
We also contain and have other makefiles.
In the Android.make, we have this one,
and in the Native Android.make, we have this one.
This one, of course, only refers to libraries that are in Froyo,
and this one refers to libraries that are actually available
in Gingerbread and beyond.
And what's important
is both of them use the same local shared library,
and that's where the primary amount of our code
will actually sit.
So we do have one problem,
and that is Android does not support
dynamically loaded libraries that are automatically linked.
We don't do dependency checking.
So we have to have a way of loading that other library
before NativeActivity kicks off.
The good news
is that we actually provide a method for doing that.
We can extend NativeActivity.
So all we're doing in our-- in NativeActivity
is creating a static member
that loads our primary dynamic library.
Now in our standard JNI-based version,
we simply load them in order.
And that's it.
With this, you can actually have two--two different activities
that share the same native code path
but target different devices.
Now let's talk about what you need to do
in order to write a game.
Well, you need to deal with the Android Lifecycle.
You're gonna deal with-- you're gonna have to have--
deal with Input, Graphics, Sound, and, of course, Assets.
And the reason I want to bring up the activity lifecycle
is it is so important.
The framework does a lot to help you with the activity lifecycle,
and I'm sure many of you have seen a diagram like this.
Someone comes in front of your activity.
It's visible. It get shut down. It comes to the foreground.
It gets restarted.
You know, all this stuff happens,
and there's always one line that has been missing
from this diagram, and that's this one.
Okay, and that happens when something in your process
actually changes the configuration.
So you're at--so look at all the different ways
your application can go through this chart.
But here's what's interesting--
There's only one in which your process actually gets killed.
Most of the time,
your native process is never going to get killed.
Your library is not going to get reloaded,
and this means that you actually have to be a little bit smart,
because Android tries to keep your process around
for as long as possible, so even if you finish your activity,
your native process is going to continue running.
It's gonna continue to get a bunch of CPU cycles.
What does this mean?
It means if you actually continue that running,
it's gonna be bad.
Now visible activities are given high priority,
so it's not the end of the world,
but it is something that is very important to note.
So what does this mean?
Be careful with static initializers.
Your library will not be unloaded
as long as your process is active.
So this means that when your activity restarts,
you need to be able to manually reinitialize anything
that you might be relying on static initializers to do.
So in general, don't use them.
Uh, also, be careful with threads,
because threads are tied to your process.
They'll continue to run even if your activity has ended.
And what that means is that people will be using the device,
and they'll find that it's slow and terrible,
and they'll look in the battery life meter,
and they'll realize that your application is the one
that's sucking up all the battery life
and then they'll go on Android Market
and they'll give you a one-star review and say that you suck.
I mean, they might do that anyways,
but at least let's not give them a reason to do it.
Um, other things to note
is that Gameplay should stop during onPause,
but your application might still be visible,
and I'll tell you, the most standard way that can happen
is if you're using In-app Billing,
you'll bring up--it'll--it'll actually bring up an intent
that'll cause an activity to show up not full screen.
Well, your application is technically paused,
but still visible,
so it might make sense to do some more stuff in onStop
than you were doing in onPause.
Still save all your state,
but maybe you don't want to trash
all of your GL variables in onPause in that case.
Finally, in onDestroy,
you want to make sure all your native code is cleaned up.
And a lot of people are like,
"Well, onDestroy never gets called, right?"
I mean--but it does.
There are definite ways in which it can get called,
especially if you're not handling orientation changes
or changes in--in, uh, in devices,
like, when a keyboard is attached or something like that,
and it tries to restart your activity.
So you've got two choices-- you can either handle
all those--all those configuration changes,
or you can be really, really vigilant about this--
one of the two.
So let's talk a bit-- just go quickly
about all the things you've got to handle.
You've got to handle input, okay?
Key handling is important.
A lot of devices have capacitive buttons,
and what this means is that we actually madea change
in Android 2 that said, instead of just relying on onKeyDown,
we actually went, and you have to get a cycle
of onKeyDown and onKeyUp.
This prevents you
from accidentally swiping into a button
and clicking it when you don't mean to a lot of the time.
It doesn't mean that it doesn't happen, but it helps.
Another thing to do is, if you're gonna do things
like handle key repeats, use the actual functions
that are built into the framework
so you--so you get the same key repeat counts
and the same behavior you would expect
from an Android application.
Touch and Multitouch handling is similar.
We have things like GestureDetector.
Use those so that your application behaves
like a non-native application whenever possible,
when it makes sense.
And then finally and probably most importantly--
dealing with Multitouch.
Now most devices now support Multitouch,
and most devices can actually track distinct points,
but you can query to find out whether or not that's true
by checking this variable--
"FEATURE_TOUCHSCREEN_ MULTITOUCH_ DISTINCT."
You can also use it as a filter in Android Market.
If you want to make sure your device
supports true 2-finger multitouch, you can.
We also add a feature in Gingerbread called Jazz Hands,
which allows you to see
if you application supports 5-finger multitouch.
So these are all--all things that you can do
to help configure your why to make sense for your user.
Now even if you don't support distinct multitouch,
you can do things, like have virtual D-pads.
You just want to put them in opposite corners of the screen
so that they will never intersect.
It is cool to handle things like trackballs.
For the devices that have them, I love it when it's an option.
And, of course, um, everyone's gonna want to use things
like sensors, you know.
When people are gonna want to take advantage
of the Xceleron or the Gyro. You just want to make sure
that it's disabled when you're not using it.
You're also gonna want to make sure to "Androidify" your game,
and what I mean is don't break the hard buttons.
So, you know, "Back" should do something.
There's nothing worse than when you're on the title screen
of a game, you hit the "Back" button and nothing happens,
and just, you know, the Pease Electric goes off
to let you know that you're hitting the button,
but the game is not responding or doing something.
Make sure it does something.
Uh, you know, Menu could-- should probably do something,
and if someone accidentally swipes the Home key,
there's nothing worse than when the game goes away.
So the result--and volume buttons are very similar.
We don't want to eat those messages.
You--you actually have to return them to the framework,
and you want to set the volume control stream.
And then again,
don't let the hard buttons bake--break your game.
You want to save your state. It's very, very important.
You know, these buttons do things that are relevant.
OpenGL--Let's talk very, very quickly about this.
The coolest thing about OpenGL and Android
is that you can actually create your surface inside of Dalvik
and take advantage of it directly from within the NDK,
and that's awesome, because, actually,
it's a lot of work
to create OpenGL stuff--
OpenGL context compatibly across Android.
And GLSurfaceView--if you ever look at the source code,
and, of course, what's awesome is that you can,
um, actually contains a whole bunch of stuff
that you don't want to have to rewrite
that works around a lot of issues
that can possibly happen in the GL driver.
So I definitely recommend using it.
Now it's not the most efficient way to multitask,
because it's going to automatically release
your context during onPause
if you call its onPause state during the onPause.
That's when it does the work for you.
You might want to call it during onStop
if you know you're gonna be using a In-app Billing--
one thing to think about.
But also with, um, Honeycomb,
you can actually say,
"I want it to preserve my state," and it will.
So we added that as a feature in Android 3.0.
And when your context is lost,
you're gonna have to do a lot of work.
You're gonna have to reload your resources--
things like textures, buffer optics, and shaders,
and your names don't even persist.
So these are all things that are important to remember
when you're using OpenGL.
The marketing way of describing this
is that Android encourages innovation in hardware.
[laughter] Galpin: [laughs]
What this means is that you have a problem as a developer.
You've got to deal with potentially different kinds
of proprietary texture compression formats,
and if you want the best quality, you will,
because your-- your other solutions
are not all that great.
You can use uncompressed textures,
but what that means
is that you are going to get a lower frame rate,
because you're gonna be using more memory bandwidth.
You can use ETC1,
but ETC1 doesn't compress all data very well.
It's mostly good for data
that does not have sharp color transitions
and it does not support Alpha.
You can use Multi-APK,
which, as we announced earlier, will be available soon,
or you can put textures on an asset server,
detect support at Runtime,
and downtoad-- download to each device.
In the previous session before this one,
we talked about how to secure-- securely deliver assets
from an asset server in order to avoid people leeching bandwidth.
So if you want to know about that,
there's some information on that.
So some--some small tips about drawing for performance.
Draw order is important, especially if you're not on a--
on a tile-based renderer,
or a-a specific type
of tile-based renderer in this case.
So definitely draw front to back.
It really helps.
VBOs are critical for performance.
You want to avoid unnecessary state changes.
And finally, you need to use power of 2 sized textures,
because if you don't,
it can either be very slow,
or it's just going to take an enormous amount of memory,
because it's just going to scale your texture up
to the next power of 2 sized textures
automatically inside the driver.
Or it's just not gonna work.
Um, and then, of course, writing compatible shaders,
and this--this slide is actually contributed by my friend Kim
over at Sony Ericsson, because he--they've done a lot of work
on this, but, uh, it's very, very important
to query for the number of attributes, and the varying,
and the uniforms that are supported
when you're writing your shaders.
And just because they're there
doesn't mean they're actually going to perform well,
but if it's not there
and you try to write a shader that does that,
it's going to crash your application, and that's not fun.
Uh, and of course, you want to avoid things like conditions
and discard as much as possible,
'cause it's really going to slow down the performance
of your shaders.
Mobile shaders are just in the beginning on their lifecycle,
and they are going to do very, very cool things,
but right now, we have to be a little cautious
when we write them.
Quickly into sound--
Android includes a bunch of options for sound.
It can be a little confusing,
so let's go through them super quickly.
We have MediaPlayer,
which is really a full-track media solution.
We have SoundPool, which allows you to play sounds from memory,
and it's actually the only way
to play compressed audio from memory.
That is not yet supported in native OpenSL.
We also have, of course, AudioTrack,
which is something we introduced in--in...
Android 1.5... [inhales sharply] I think,
and, uh, it, uh, allows you
to play PCM audio only with minimal latency,
and it's really meant for applications
that are generating their own audio.
And finally, we have OpenSL, yes,
which does require Android 2.3 or higher,
but it combines the capabilities of MediaPlayer and AudioTrack,
and it allows you to do all of your audio from native code.
And it--and the interface is actually a little nicer
in terms of generating audio,
because it actually calls you back when it needs a buffer.
So it's a very, very straightforward interface
to use.
And I'm sure, going forward, it's gonna become
even more and more powerful and important.
So let's talk quickly about assets.
One of the things that people want to be able to do
is pull resources and assets out of their APK file.
And I went to the Android team, and I said, "All right,
"there are many ways we can do this.
"What is the way
that is most likely to still be compatible in the future?"
And they suggested this method.
So what we're going to do is get our APK file name,
uh, using this SourceDir.
Then we're going to use the AssetManager code.
And that AssetManager code allows us to get something
called an AssetFileDescriptor.
All an AssetFileDescriptor is
is a standard native file descriptor
that includes extra parameters for offsetting length.
We can read them, open the APK file from that offset,
and there we are.
We're reading our--our resources right out of native code
without any additional overhead,
which--and the overhead we're trying to avoid,
of course, is having to buffer things
between Dalvik and native code.
You can, of course, use a native Zip Utility
in a very similar way.
It's a little--it's a little less future-proof,
but it's probably going to work as well.
Um, and it's probably gonna be faster,
as long as you do things like cache the Zip directory.
And then finally, of course, you can use native AssetManager
on Gingerbread and beyond.
All right, my tongue twister slide--
"Troubleshooting, Tips, and Tricks:
Targeting tricky, trying tasks."
So Threading, Dalvik, and the NDK--
all context are per-thread.
You've got to be careful to check your threading.
If you take that same context
and reuse it across multiple threads,
bad things will happen, because the JNI context
tries to do smart stuff for you,
like clean up objects that you're not using anymore.
So you want to use one JNI context per thread,
or you want to take advantage of a game loop
that actually takes your messages,
threads them into another thread.
It's also useful if you're calling the framework,
'cause many of the framework classes don't like to be called
from anything other than the UI thread.
As far as performance, you can check for older hardware,
and there's multiple ways of doing that--
checking for ARM7, checking for support for OpenGL 2.
And then you can use Fallbacks.
You can use things like 16-bit textures.
Um, you can actually convert them on the fly
using BitmapFactory.
You can also use a smaller render target
using SurfaceHolder.setFixedSize.
Of course, there are other ways of doing that within OpenGL,
but this actually does it at the optimal place,
which is at the compositor.
Pixels are very, very expensive,
especially on tablets, so this really will help,
but if all else fails, we do have market filters,
such as Device Availability, which was announced today,
platform release, GL version, and instruction set.
And you can--you can filter on instruction set
by only building a library
for ARM5 or ARM7.
And here are just general tips.
Start small. Don't make me download a big file.
Use the backup/restore service,
can support multiple control schemes,
install to SD-- I'm not gonna go through these,
but you should all consider these, things like profiling.
You know, make the game render up and do better stuff,
if I--if it detects that it's running on a cool system.
Hooking into social networks is, of course, really important
for viral spreading of your application.
Building a live wallpaper is so cool
and so few applications do it.
But, like, like, isn't that awesome?
You get to own the user's home screen.
I mean, what other platform allows you to do that?
That's so awesome.
Um, analytics--definitely find out where your customers are.
Find out how they're using your title.
Like, this is awesome, awesome information.
Find out where--use--use it to do things like finding out
even where someone is dying in the middle of your game.
Like, you're like, "Hey,
"that level's way harder than I thought it should be.
What's going on?"
Or, "Only on this device people are having problems."
All that kind of stuff you can find out.
And, of course, In-app payments, we went through that
in the last presentation I just did.
We are now seeing amazing traction in app payments.
If you look, we just added
a new category in Market for top-grossing applications
and three out of the top five
are top-grossing only through In-app payments.
So some tools--I'm gonna throw these up here very quickly
so that when this slide-- when this is played back
and put on--on, uh, on the web, you'll be able to get these,
but this is some of the stuff we've talked about,
and, of course, we have some performance tools.
Each one of the chips at manufacturers
has now come out with performance tools,
and they can be very useful.
Some of them require a routed device.
Now we've got a very short period of time for Q&A.
So, uh, I went through those a little fast at the end,
so if you have a question, this is the time to stop me on it.
And, uh, thank you very much.
[applause]
Galpin: And I recommend everyone
stay around in the room after this,
because, um, there's gonna be something special
for all of you, so... [chuckles]
Free cake? That sounds good to me.
All right, next.
man: Hi, uh, I work on the open source port of Freeciv,
if I can just give a quick shout-out.
Galpin: Awesome. man: So it's--it's open source.
It's, I think, one of the largest NDK applications
you can--you can freely get the source to.
Um, there's a lot of terrible ideas in there,
but hopefully, there's one or two, uh, good ones.
Galpin: Cool. man: Uh, I had a quick question.
It was something I'd seen when--when loading a resource.
One--one of the goals of that project was to be able
to drop my app, even let it be killed and have it, like,
spring back as quickly as possible to where you left off.
Galpin: Right. man: And so I spent
a lot of time optimizing how I loaded my art assets.
Galpin: Mm-hmm.
man: I found something really counterintuitive.
It seems JPEGs load about three times faster than PNGs.
Is that, um... sensible? Galpin: Is that--
Is that based on the file size? 'Cause--
man: Uh, it--what I found was when benchmarking this
on a Samsung tab... Galpin: Mm-hmm.
man: was only about 1 millisecond per--per file
was taken up in file I/O. Galpin: Mm-hmm. Mm-hmm.
man: And that was effectively the same
whether PNG or JPEG.
Definitely the PNGs were larger. Galpin: Hmm.
man: The JPEGs were smaller.
It was the actual--apparently, it was the actual time
spent, like, converting from the compressed stream
into a bitmap.
Galpin: Hmm. Interesting. I-I haven't experienced that.
But, you know, I-I--
Ni-Lewis: PNGs aren't on the no-compress list, are they?
Galpin: No. Yeah, they're-- they're both--they're both
on the no-compress list, yeah. Ni-Lewis: Okay, I thought so.
Galpin: So that's-- the only thing
that would cause that normally is that--is that, you know,
in the zip file, uh, you know, things,resources
can be compressed, but both of those files
are automatically not compressed.
I'm--I'm obviously pretty surprised you're seeing that.
But I-I--you know, I don't actually know the answer.
That's a cool thing to note. man: Yeah, and--
Shift--even shifting that around, it didn't, like,
if--if I was on the SD card, the only thing
is if it was on the SD card, it would seem to take
about twice as long as if it was on flash memory.
Galpin: Yeah, see, that--that would--that would really make me
think that it actually was, on some level, file I/O.
man: Yeah. Galpin: It might be even, uh,
just--just because, uh, uh, that would be my first--
my first guess is it's entirely I/O bound, but, um--
man: Right. But, uh, three times, and I mean, you know,
these weren't very large files. Galpin: Yeah.
man: I mean, it was--it-- I don't know.
And it could also be--I know that it's the OMAP platform
and they have optimized JPEG decoder,
so maybe out floating some of that to a DSP.
Galpin: I don't think so, but that's, uh, that's it.
I think--I think it's-- Ni-Lewis: And it could be,
you know, loading it through and optimizing code paths.
There are two different code paths for it, but I-I--
Galpin: It could be loading it through--yeah.
man: Oh. Yeah, I did actually try compiling the libJPEG
and using, like, instead of loading it from, um, Java,
pushing it into a texture, and then pulling it up
through there. Galpin: Yeah.
man: I-I tried the libJPEG thing
and--and it even seemed like using just
a bog standard compiled libJPEG performed better than PNG,
but performed worse than--than pulling it up through Java.
Galpin: Oh, interesting. Cool. man: All right, well, thank you.
Galpin: No, that's interesting. That's good to know.
Ni-Lewis: It's a mystery. Galpin: It is.
man: Um, so question--
Uh, I have downloaded NDK using the JNI,
but I find out that debugging is extremely difficult.
So I saw that we can do the complete,
uh, Android application in C and C++ only,
without using Java.
Uh, so that makes the debugging much easier.
Can you show me, uh, some way in the future
we can support only C and C++ for entry application?
Ni-Lewis: Ging-- with Gingerbread,
you can support that,
but let me sort of lay things on the line for you.
The, um, the debugger for C and C++
is, uh, honestly, a little flaky.
Um, every single version gets so much better,
it's just amazing. man: Mm-hmm.
Ni-Lewis: Um, but it also depends a lot
on what you're using with it.
Um, so for instance, were-- were you using, like, stock GDB,
or were you using Eclipse or...
man: Uh, many use, uh, Eclipse, yes.
Ni-Lewis: Okay, so right now,
um, if you--if you use Eclipse,
and you either--there's-- there's instructions
on the internet for how to get it to work with--with any GDB,
or you can also, um, use NVIDIA's plug-in for Eclipse.
It sets up the debugger for you.
The--the--the only disadvantage to that
is your device has to be rooted, uh, right now.
Um, hopefully, they'll fix that.
Um, then actually, it's-- it's fairly easy to debug.
You can bounce back and forth between Java and, uh, JNI,
right--oh, and your-- and your C++.
Galpin: Yeah, there's--there's no--well, I was gonna say
the main answer is that there's really no reason
why--why you can't debug native code
that's being called from--from JNI.
It actually works--it works, as far as I can tell,
just as well as-- as fully native code.
Ni-Lewis: Yeah, and--and the bad news is
going to full-on C++ would not help anything,
because it's not the--it's not bouncing back and forth
between Java and C that's the problem.
It's just the debug layer, and C is--is a little flaky
with Eclipse.
man: Okay, thank you. Galpin: Mm-hmm?
man: Uh, I have a question regarding Google TV
in the NDK. Galpin: Mm-hmm.
man: And that, uh, after talking to the Google TV guys,
the NDK is not supported for, uh, Google TV.
Galpin: Mm-hmm. man: Is that ever going
to change and... Galpin: I think that's kind of--
I think that's kind of up to the Google TV team.
Um, as-- as a C and C++ developer,
I would love it to change, but, uh, but that's--
that's--that's up to them.
Ni-Lewis: Yeah, actually, you're not the first person
to answer--to ask this question.
Um, and I've worked closely with the--the Google TV team.
They--basically, they're not making any announcements.
Um, they haven't ruled it out one way or the other.
Um, but they--they have said on many occasions
that they understand
that it's a-a desire that a lot of people have.
man: Yeah, especially with regard
to, uh, some sort of, like,
Xbox live arcade style gameplay.
Galpin: Absolutely. No, it's totally--I'm there with you.
Ni-Lewis: We--we think that would be cool, too.
man: So--so with each version,
indicates you guys are really progressing...
[speaking indistinctly]
so good direction.
But the question is about accessing the MediaPlayer,
the video player back, actually, from native code, so...
Galpin: Yeah, so--so right now,
there's--there's no way to do it, uh, directly.
Um, I'd love to see that change,
but, uh, that's still the way things are,
so you still have to go back
and bounce back into, uh, into Dalvik to do that.
man: But that's on your road map, right?
Galpin: [laughs] man: [laughs]
Galpin: I would really like it to happen.
man: [chuckles]
man: Uh, first of all, let me say thank you
for multithreaded GDB and Gingerbread.
Uh, other--before, it was just a pain
having to call things on the main thread.
Galpin: Yeah. man: Um, I actually, um--
Galpin: Thank David-- thank David Turner
and the--and the--and the Dalvik
and the--and the Android, uh, Kernel team, actually.
They made that happen. They're both awesome.
man: I actually don't work on games, but I imported
an extremely large engineering app from C++,
um, and we needed, um, we were using the CrystaX before,
because we needed wide string. Galpin: Yeah.
man: And we needed, like, a full SDL,
but for Gingerbread, we've actually switched now
to using new--the SDL, but the problem is,
the, um, the one that's part of the NDK5
doesn't export any of the actually wstring functions.
Um, if you--it builds, but it doesn't link to wstring.
Uh, and I tried really, really hard,
and I looked around, and eventually, what I had to do
was I had to download the NDK source code
and change the build scripts to get wstring to export,
and it's like a meg and a half bigger,
and it's fine, um, but actually, my question is this...
Galpin: Yeah. man: Um, so I changed that.
Do I have maybe--I may need to ask a lawyer this,
but do I have, like, an affirmative GPL obligation?
'Cause I only changed the build scripts.
I didn't change GCC, but I'm using my own version, I guess,
now of the NDK, not the source-- not the one that was...
Galpin: Well, all of, I mean, I'm--I'm not an attorney.
man: Yeah. Galpin: But my understanding
is all of Android is licensed under a, uh, license
that doesn't force you to contribute your changes
to all of--all of-- man: Well, 'cause I didn't know
if GCC--yeah. Galpin: Yeah, if you're using
a GCC stuff, you know, it's interesting.
I don't actually know what the answer is--on that.
But I'm surprised the wide string stuff isn't working.
You should file that as a bug, because I just spoke to it.
man: Okay. Galpin: So that was one--
that was one of the desired changes.
Ni-Lewis: That--yeah, well, we-- I talked to David about that.
Galpin: Yeah. Ni-Lewis: Yeah, I think--
now he said that he actually couldn't get the command line,
that there were like two mutually exclusive options
where wide string didn't work. Galpin: Yeah.
man: Uh-huh, there are, and I actually had
to HardCode one of them. They're like boolean opposites.
Galpin: Yeah. Ni-Lewis: Okay.
man: And I went into one of the makefiles, and I figured,
yeah, and I basically had to manually override it.
Galpin: Right. Ni-Lewis: And this
is really good-- this is one of the things
that you should speak up, uh, on the forums or e-mail us about.
man: Okay. Galpin: Yeah.
Ni-Lewis: Because--because as you can see--
so really difficult build, uh, solution that ends up
making your code a mega and a half bigger.
Galpin: Yeah. man: Mm-hmm.
Ni-Lewis: And maybe isn't all that desirable by most people.
man: Yeah. Ni-Lewis: You know, you can
probably see why that choice was made.
man: Yeah, absolutely. Ni-Lewis: But if we've got more
feedback about that, more data points, you know,
that might be something we want to do--support in the future.
Galpin: Yeah, absolutely. man: Thanks.
man: Okay, quick one. Galpin: Mm-hmm.
man: Uh, any plans to access the camera API through the--
through native code? Galpin: [sighs] Yeah,
it would be--it's another-- it's another "nice to have."
I think, you know, I think ultimately, here's--
here's the issue.
I mean, the--the general-- the general thought
is if there isn't a strong performance reason,
for the most part, um, you know, the--the goal is to make it
something that you have to get through, uh, through Dalvik,
because that is the primary--
the primary platform for development for Android.
That being said, there's a lot of people
who want to do really cool stuff with the camera that,
if you had a native interface, you could do it a lot better.
So I-I think that's--I think that's an important one
to--to bring up, and--and again, keep requesting it.
Keep complaining about it. Hopefully, it will happen.
man: Thank you.
Galpin: All right, so that being said,
uh, I'd like to, uh, to mention,
uh, that, uh, how many of you guys here
are game developers or want to be game developers?
Okay, well...
Ni-Lewis: Everybody wants to be a game developer.
Galpin: Uh, well, yeah. Well, you know, it's--it's--it's--
it's a tough job.
Um, so we have something very special for you.
We've, uh, Sony Ericsson has come today,
and, uh, they are here
to make sure that every one of you
has the ability to develop really, really great games.
And so they want you to have a device
that has support for a modern GPU,
has the support for a lot of different controls.
They want to see what you can do with it.
So, without further ado, on your way out,
you will be getting a card,
and you will be able to exchange that card
for your own--very own Xperia PLAY.
[cheers and applause]
Galpin: So...
our--our only ask,
and it's that we want to make sure
you guys do really, really cool stuff with it.
If you do,
uh, and you're gonna want to go a couple things--
one is you're gonna want to go to sonyericsson.com/developer,
and--and they've written a lot of articles about
how to get to the special features of that device.
The second thing you're gonna want to do is,
if you do something cool, go to standout.sonyericsson.com,
and let them know that you've done something cool,
because they're always looking for people
who are doing cool stuff with the Xperia PLAY.
So, uh, with that, I hope you all enjoy it,
and I'm very excited
and a little jealous for you all.
And, uh, and there we have it.
Ni-Lewis: Thank you. [cheers and applause]
Galpin: Thank you.