Cross-Compiling Android Applications to the iPhone

>>
LEA: Today, we have Professor Arno Puder from San Francisco State University. Professor
Puder--prior to his current position at SFSU, he worked for AT&T Labs Research and German
Telecom. His research interests include middleware, ubiquitous computing, and applications for
sensor networks. Today, he is going to talk about Cross-Compiling Android Applications
to the iPhone. And I'll give it to Professor Puder, then.
>> PUDER: Okay, thank you, Lea, for the introduction. And, well, thank you for inviting me to talk
about my research. Smartphones had become very popular and particular are those two
that I've mentioned on my slide here, on the title slide; Android and iPhone. That certainly
created quite some buzz. And as more and more of the smartphone are released, of course,
the problem also increases in terms of deploying your application on different smartphones.
So the technology I'm going to talk about today is helping you porting applications
from one smartphone to another. And in particular, I will be talking today about, well, as the
title suggests; Cross-Compiling Androids Applications to the iPhone. Now before I go into any technical
details, I wanted to just briefly introduce the team. Now of course with any open source
project and XMLVM is an open source project, you stand on the shoulder of giants. So XMLVM
as an open source project, it's making use of over half a dozen other open source projects.
But it almost boils down to the people. So within our project, there are a number of
people who have made significant contributions to XMLVM. So the core team besides myself;
Sascha Haeberling who is actually another Googler, he works out of the Zurich office;
and Wolfgang Korn, who works for a consulting company in Germany, blueCarat. And blueCarat
is also a corporate sponsor of our project. Now, as a growing open source project, we
also have now a growing number of people who make contributions to XMLVM. And at least
as of today, I list those names that I've--of people who have done some working on our project.
Well, a bit of background. I guess it goes without saying that smartphones certainly
are getting very, very interesting. If you look at the growth rates, so they have double-digit
growth rates, and that I'm sure is going to persist for quite sometime to come. Now if
you are an application developer, then you are faced with the problem of rewriting your
application for the various platforms. And that can consume a lot of time and resources.
Now, while the programming environment is a little bit different, the smartphones do
have similar capabilities when it comes to their hardware features. So, again, the problem
here is that that you as a developer are faced with this problem of deploying your application
cross-platform. And on the next slide here, I've put down some, well, a kind of a little
snapshot here. So on the first column, I have the HTC G1. I have, on the second column,
the iPhone 3GS. And the third column, I have the Palm Pre. Now, Android is releasing more
and more devices and Google has just recently released its Nexus One. So, of course, even
with an Android, there is now a multitude of different devices out there. But, again,
like for the slide here, I guess, the important point is if you look at the upper part of
that slide, I gave you a little background on the hardware details on the various devices.
And you can see, there are almost, at least, very similar. And if I keep on adding more
and more devices here, they probably will be the same in terms of speed, in terms of
CPU, in terms of RAM. But then if you look on the bottom half of the slides, you will
see that they're actually quite different when it comes to how we have to develop an
application for them. So, I mean, you as a developer, of course, you look at the smartphone
from a SDK perspective. So how do you actually write an application for the phone? And if
you look at Android--well, Android is using Java as the language to write an Android application.
And the GUI is based on specific Android API that you make use of writing an Android application.
You have virtual machine technology that's allowed on it's--on their phone and everything
is open source. Well, I mean, there's a little bit of a caveat of course. There are some
proprietary parts that Google does not turn into open source. But at least for the Base
SDK, everything is open source. Now if you look at the iPhone, that's completely radically
different. So we have a language called Objective-C. And I will say a few more things about Objective-C
on the following slides. Furthermore, you have a proprietary API called Cocoa Touch
providing iPhone Applications. And of course the complete tool chain is proprietary. Actually,
you need even a Mac platform in order to write an iPhone application. Now I also want to
say a few things about the Palm Pre. Actually, it's a device I like very much. I think it's
unfortunately a little bit underrated. It should be given a bit more attention if you
ask me. But if you look at how to write Palm Pre applications, they actually have decided
to make use of WAP technology. So if you want to write a native Palm Pre application, you
end up writing HTML, CSS, and JavaScript. Almost as if you are writing a web application.
So again, like the point of that slide is here that while the phones are somewhat similar
on the hardware side, they are radically different from the way you actually write applications
for these phones. So the propose solution that I'm going to go into some detail here
in this presentation is that we say, "Well, as kind of a frame of reference, we take Android."
So we expect people to use Android and we do that for various reasons. Well first of
all, I mean, I'm in education business. And there are certainly a lot more skill set out
there for Java than there is, for example, for Objective-C. So if you look at how much
innovation you can achieve, certainly, I would argue that basing on Java is certainly a smarter
choice than Objective-C. So I'm not looking at the current number of applications you
have currently in the App Store, I'm looking overall at the skill set that they have out
in the industry. The second thing is that Android was built from day one or it was designed
from day one to serve heterogeneous set of devices. So in particular, Android copes very
well with different screen resolutions. And the iPhone, for example, is only doing that
now. Apple has recently announced the iPad and only now Apple has faced with the problem
of having two different devices; the iPhone and the iPad that have different screen resolutions.
So only now actually they are beefing up there, enhancing their SDK to also support different
screen resolutions. So from that perspective, it makes a lot of sense to take Android as
the frame of reference. And then what we do in our tool chain, that kind of--is also the
outline for the rest of this presentation. So the first step that we do is--oh, that
we did rather--is to design a Java API for Cocoa Touch. Remember that Cocoa Touch is
based on Objective-C. So the first time will be to create a Java abstraction on Cocoa Touch.
The next step is going to be to cross-compile Java to Objective-C. Since we're starting
from Androids then we want to go over to Objective-C, we have to cross-compile that somehow. And
maybe I should also stress this point because the iPhone or Apple rather prohibits by licensed
agreement the use of virtual machine technology on the iPhone. So you cannot run a virtual
machine on the iPhone. And--but by cross-compiling, we actually end up with native--a native application
that does not require a virtual machine. So our toolset is within the legal framework
of the Apple SDK. Now, the last step that we have to do is to deal with the API mapping.
So if you write an Android application that has different kind of API than the iPhone
obviously, and the last step that I will be talking about today is how we deal with that
through what we call the Android Compatibility Library. So as a programmer, as a geek, the
first thing you do in any program language is "Hello World". So what does "Hello World"
look like for the iPhone? Well, as I mentioned before, the iPhone is based on Objective-C.
So if you want to write a native application for the iPhone, you have to do that in Objective-C.
Now, I tell my student sometimes that Objective-C is like the Latin--are like the language Latin,
you know. There's exactly one country in this world where Latin is being spoken, that is
the Vatican, of course. So Objective-C is used inside of Apple, but not used much outside
of Apple. And it is a dynamically-typed language. It's a strict superset of the C programming
language, so every legal C programmer is also a legal Objective-C programmer. And what you
see here is, again, is the "Hello World" program, and if it's at any kind of GUI programming
then probably you will make sense of it rather quickly. One thing you often notice or the
first thing you often notice when you look at your first Objective-C program is that
you see all these square brackets. And these square brackets denote method invocations.
So whenever you see a square bracket, the first argument is actually then the target,
which can either be a class object or an instance and anyhow, the list of named parameters that
you are sending to that object. So for example, if you look at the first thing here, "[UIScreen
mainScreen]", basically, "UIScreen" is a class of the Cocoa framework, Cocoa Touch framework,
and you ask it to give back the "mainScreen", so what you get back is a pointer to this
object. And you can actually see here C shimmering through a little bit because you still have
these pointers that you have to deal with in Objective-C. Now, if you look at to the
rest of this program, I think you can probably make a sense rather quickly if you done, for
example, some Swing programming. So if you look at class "UIWindow" that is being used
here a little bit down. That kind of corresponds to a "JFrame" if you have--in the Swing programming.
"UIView", it might correspond or it might take an analogy to "JPanel". "UILabel", then
again, correspond to something like a "JLabel" in Swing. So with that, I think you can probably
understand what is happening here. Just maybe a few side notes here. First of all, the entry
point of an iPhone application is a method that you have to overload that is called "applicationDidFinishLaunching".
So that is--I didn't come up with that name. That is the official API as defined by the
iPhone SDK. So that is going to be the first thing that's going to be called of your iPhone
application. Of course, you have to derive your application class from base class of
"UIApplication" in order to overwrite this method. But that will basically then kickoff
your program. That's the first thing I want to point out. The second thing I want to point
out--I already mentioned that Objective-C is a superset of C. So that's why you have
these pointers. I want to point out the second line here of my implementation. And I'm not
sure if you can see the mouse here. So basically, I asked the screen object for its resolution
and I get back something of type "CGRect". Now, you will notice that there is no pointer
here. Actually "CGRect" is a simple struct. And unlike C++, struct is not a class. A struct
is just simply a value type. So what is done--what is happening here, actually you are being
given back a struct that is copied by a deep copy mechanism here. I want to point that
out because Java, as you know, does not have value types. So when we do API mapping, we
have to also kind of breach this--well, you might call it a problem when we have value
types in Objective-C, we have to represent it somehow on the Java side. So this here
is now the Objective-C version of "Hello World" for the iPhone. So the first thing that we
have done in our framework is to come up with a Java API. So on this slide now, what you
see is a Java program that is basically using our API mapping for Cocoa Touch, but it's--well,
first of all, 100% Java and--but it has the same semantics. So with all I've said on the
previous slide, I guess, you can immediately understand what is happening here. So we've
cut the naming convention. So you see? We have to derive your application class from
base class called "UIApplication". You have to overwrite a method called "applicationDidFinishLaunching",
and then also for the other classes; "UIScreen", "UIWindow". We just have taken the same names,
of course, as we have found them on the Objective-C site. So we are trying to mimic the API from
Cocoa Touch that we see on the Objective-C side as closely as possible on the Java side
to make it more easier for a programmer to make that transition. So ideally, you can
even go to the Cocoa Touch API documentation and then kind of understand what these classes
are doing. But of course, we have taken some design decisions here in terms of staying
closer to the nature of Java. So we have made use of strongly-typed interfaces for one thing,
so wherever we see something that never get typed on the Objective-C side, we'll make
use of Java interfaces. And also like, for example, your constructors. We are using constructors
a little bit more of the way they are supposed to in Java. And it's also a little bit different
the way it's done in Objective-C. So if you take a look at this slide here now, so if
you take this Java source code here, then basically what happens in our tool chain is
that we cross-compile this Java source codes over to Objective-C source code. And how we
do that, I will explain on the next few slides. But once we have the cross-compile to Objective-C
source code, we link it against a little library that we call the Objective-C Wrapper Classes.
And then what we end up with is a native executable on the iPhone. So there's no need for a virtual
machine. So you can deploy that on your iPhone and, again, since we are within the legal
frameworks of the Apple SDK, we can also push it on the App Store, of course. Now, if you
remember, the--even a little "Hello World" application I had on the previous slide was
making use of a class called "UIScreen", "UIWindow", "UILabel". Now, what it can do is--and that
is what--I have no--on the slide here--what it can do is it can actually provide a Java-only
implementation of these classes. So if you have a class like "UIWindow", instead of just
having a skeleton, why not just implement it also based, for example, on Java 2D. What
you end up with is a 100% pure Java version of your iPhone application that you can run
on any platform. So any platform that supports Java, you can now run. Basically, it's out
own Java-based iPhone emulator that allows you to run your Java-based iPhone applications.
And I want to switch over for the first demo to explain how this works. And I'm in Eclipse
here. And I'm going to launch now a little application that is kind of our trademark
application called Fireworks. And it does--not a whole lot. It just switches into full screen
mode and then it does the little fireworks. And I have my iPod touch here, so if I call
fireworks here on this device then, of course, you see the same thing. Now, there's one little
gimmick to the application, when we turn it around, we use the accelerometer so you will
notice on the device here, if you can see that the sparks always follow gravity. So
no matter how I turn the device, the stars will always go to the bottom. So--and we have
mimic this behavior in our own Java emulator here by the scroll bars. So if I modify these
scroll bars here by feeding different values into the accelerometer, then you can see that
the stars start flying into different directions. Now, again, I want to stress what the application
you see here that--the thing is that what is called the iPhone simulator. It's a 100%
pure Java application. You can run that on the Windows, in the Linux. And the application
that you have seen on my iPod touch is just the cross-compiled version of the application.
Of course on the device, the classes are linked against the Cocoa Touch classes so to yield
the behavior that you've seen. There's a little bit more that's kind of a fun thing. So here
actually I have on my iPod touch, a little remote control application. And what I can
do is I have created a talk network between my iPod touch and my laptop here. So what
I'm going to do now is I'm going to punch in the IP address of my laptop. I hope I get
this right here. So what happens now is my iPod touch acts as a remote control for my
Java-based emulator. And what happens is when I turn my iPod touch, you will see that--well,
you can see that the scroll bars kind of turn by magic. So basically the readings from my
iPod touch are being sent via HTTP request to our Java-based emulator. So when I turn
the device, my iPod touch, it kind of--if I turn it upside down, my device and the stars
in my Java emulator start flying up. Now, this application that I've used here on my
iPod touch, we've actually implemented using our own technology. So what I can do now is
I can go back to Eclipse and I can just simply run the second instance of our Java emulator.
And what you see now here is actually the interface I've just seen on my iPod touch.
So I still have the fireworks running here in parallel, but what I can do now is since
I'm running both emulators on the same laptop, I can just simply type in the loop IP address.
I can turn this on. And if I now use the controls on the one emulator, you can see that it reacts
over on the second emulator. So if, for example, I go up here with X, it starts going left
and right. So we can see that actually for our Java-based emulator, we have already pushed
it to some extent here, so all that you see here, again, on the second emulator is just
a Java implementation. So we have mimicked a little bit the look and feel of the iPhone
by providing a Java 2D implementation of the Cocoa GUI elements. So let me go back to my
presentation. And--so let me talk about some of the challenges that we're facing. Before
I actually tell you how we do certain things, let me tell you, first of all, the problems
that we have in trying to do so. Well, first of all as I've mentioned before, as a design
goal, we try to mimic the Java API as closely 1:1 as we see from the Objective-C API, but
we make use of strongly-typed interfaces so just you stay in the spirit of the Java programming
language. Now when it comes to challenges, there are challenges related to the language
cross compilation, there are challenges related to the API. And let me first talk about the
Objective-C challenges. Well, first of all, Objective-C has no name spaces. Objective-C
does not permit method overloading. So they allow method overwriting but not method overloading.
So a Java supports both. So when you cross-compile, you have to deal with that somehow. The version
of Objective-C that's being used on the iPhone does not have a garbage collector. So if you
write an application, you actually have to make use of a reference counting mechanism
that is given to you by the iPhone framework. But it's your responsibility as a developer
to actually correctly retain/release object references. So since we don't have a garbage
collector on the iPhone, what we do is, actually doing our cross-compilation, we insert these
retain and releases during the cross-compilation phase. Now of course, if you're familiar with
garbage collection, you will know that if you do reference counting--reference counting,
although, it's more efficient than garbage collection but it does have some limitations.
So, for example, if you have a cyclic data structure that cannot be reclaimed by reference
counting, so that is one of the limitations that we have right now on our tool chain.
So if you have a cycle in your data structures that you build up, then you need to break
that cycle manually by assigning another point somewhere. Once you do that, of course, then
we can reclaim memory. But if you just have a cycle in your data structures, at this point,
we cannot garbage collect that. Now, the challenges we have on the API level, well, they're quite
a few. Well, first of all, the Cocoa API is not as cleanly designed as you might think.
So for one thing, it's not always object orientation that they have used in designing the API.
So you actually will see quite a few C functions there for doing certain things. So, for example,
I've--there's one here, "CGColorCreate", it creates a color object but that actually is
a C function. So you have to provide it the RGB values, and then what you get back is
something that it can use as color value. But this function here is actually a function
in the C programming language sense. So it's not--like I said, a method of a class, as
you might--would expect in a clean object-oriented design but it's just a C function. There's
a question here? >> What is that--particularly the worse?
>> PUDER: I wouldn't say it's worse, it's just--I mean, it comes from historic background,
so of course, the Quartz framework has been there for quite sometime and...
>> I mean, does that--they produce some sort of challenge to the cross-compilation [INDISTINCT]?
>> PUDER: Okay. The question is does that creates challenge in cross-compilation? Well,
it does because, remember we cross-compile from Java to Objective-C, so whatever we see
on the Objective-C side we have to create some Java API for it. So since there are no
functions in Java, we have to, you know, create some set of method of some class. So we have
to basically turn this more procedural interface into an optical interface. So that's the challenge
that we had to, you know, deal with. And I'm not saying that what Apple has done is as
bad here. It's just legacy basically, you know. The other things I'd already mentioned
before, like value type. If you look at "CGRect", that is actually not a class, it's just simply
a value type. So there are no value types in Java. So I didn't point that out in that
slide earlier, but you probably, you might remember that actually there was a Java class
called "CGRect". So, again, value types don't--they don't have something equivalent on the Java
sides. So, here, we again map it to a proper Java class. But during the API mapping, of
course, we have to map a Java class to a value type on the Objective-C side. So that happens
inside these Wrapper Classes. The next thing that kind of cause us some headaches is that,
well, at least, the design of the Cocoa API is using some old little tricks that people
might remember, the older people in the audience at least from the C programming language so
that if you need additional output parameters, there's a common trick to have a pointer to
a pointer basically to pass back additional outputs and arguments. Now, Java only supports
call by values so you cannot do call by reference. So again from the Java API and design perspective,
we have to deal with that. And what we have done here is to create some holder classes
that basically are just act as a wrapper around an output argument. And the last part is again,
the delegation that is heavily being used in Cocoa Touch. And as I mentioned before,
here, generally, we have turned that into a strongly-typed Java interface, also to stay
truthful to the spirit of Java. So let me explain to you how we actually cross-compile
Java to Objective-C. And the first thing is that we do not have a Java Source Code level
cross-compiler, so what our tool chain, what our--what XMLVM actually cross-compiles is
not Java source code, but Java bytecode. And more specifically, we are making used of the
Dalvik virtual machine or the DEX instruction set. So you might know that Android is not
making used of the standard JVM but Android have chosen their own virtual machine called
Dalvik. And the bytecode instructions are called DEX or DEX instruction set. The big
advantage of DEX is that it's a register-based instruction set. So the Java virtual machine
is based on this DEX machine but Dalvik is based on a register machine. So we're making
use of that. And the first thing that we do in our tool chain is to take the Java source
code and turn that into an XML file that where the implementation is represented by the Dalvik
instruction set. So to show you how this works. On this slide here, you'll see a very simple
Java class. So you see a class called Calc. It has one static member. One static method,
it's called Add. It just takes two integers, and all it does is just simply return the
sum. So it'd returns "x+y". So in our tool chain, what happens is well, first, you just
take a standard Java compiler to turn it into a class file. And then by piping it through
our tool chain, and again by making use of the open source tool that we've access to
from Android, we turn that into an XML file that just simply represents the Dalvik instruction
set but marked up through XML. So what you see on the next slide is basically this class
Calc but now marked up as XML and making use of the Dalvik instruction set. So it looks
a little bit--a little bit complicated, but if you take a closer look, they're actually
not that--they're not that difficult to understand. On the top level, you have--you have a tag
called "class" that is your class definition. Then you have a method that is our--the "@method"
we had before. We have attributes to denote the various properties, the modifiers that
you have for this method like, in this case, it's a static method. You have a signature
so you can clearly see that this method takes two integers as input. It takes one--or it
returns one int as a result. And then most importantly down here, you see that in the
code tag "code", you actually have the DEX instructions that make up the implementation
of method "add" by means of Dalvik instructions. And if you can see, actually, there are only
two instructions here. There is an "add-int" instruction and there is a "return" instruction.
Now, the "add-int" instruction makes reference to three registers; vx, vy, and vz. And what
happens inside the Dalvik virtual machine that the certain registrars are being initialized
with the input argument. So actually by convention, and I won't go into too many details here,
but register number one and register number two correspond with the input arguments x
and y. So, basically, what this one instruction "add-int" does here, it will add the content
of register one with register two, and it will store the sum in register zero. That's
exactly what my Java class was doing that I had on the previous slide. And the second
Dalvik instruction here, then says a return, and we're returning the content of register
0 as the result of this method. So that's kind of like a little inside glimpse into
what actually happens inside of Dalvik that runs on any Android phone. So--but why do
we convert this to XML? Well, that actually is also where the project XMLVM gets its name
from because we're using XML technologies for--than generating high-level languages.
So in particular, what we do know is we're using XSL stylesheets for the code generation.
So in the next slide, I give you one example on how we map this "add-int" instruction to
source code again. And what you see on the bottom here is now the--an XSL template that
will always fire--will always match whenever it comes across the "add-int" instruction
that I explained on the previous slide. But then what happens is that whenever the XSL
processor is coming across that "add-int" instruction, it is going to omit this code
that I have here in the--as the children of this template. And basic--the code that will
be generated is what I have here in this yellow box here. So basically, this one "add-int"
instruction is going to result in this here; "r1.i + r2.i" and that is being assigned to
"r0.i". Now, this registers here; r0, r1, r2 are based on the union that is of type
XMLVMElem. So because a register in Dalvik can hold different data types, it can hold
an integer, it can hold a float, for example. So--and this union here represents all this
different possible types that can be used toward in a register. And by--since we have
the instruction "add-int", we know that what is in those registers are integers, that's
why we are accessing the value through the ".i" discriminator. But you can see in terms
of code generations, so for each Dalvik instruction, we have like a very small, very easy to understand
template that will then basically generate code in the target language. Now on the next
slide here, I'll show you the results. If you take all our templates and you if you
run it through the XML file--I had two slides earlier--then, here is the Objective-C source
code that you would end up with. And, again, you see, of course, the other templates that
define all these registers here. So we have two registrars. They are all of type XMLVMElem
as I explained before. You can see here the "add" method. And here, you can see we're
actually using name mangling to cope with this problem that Objective-C does not allow
overloading. So we have our two input arguments, x and y which here, again, on bytecode level,
are referred to as n1 and n2. We initialize the input parameters with the registers. Here
is the one line, so the addition data is basically the code that was contributed by the XSL template
I showed on the previous slide and then the return statement then will return the result.
Now, of course, if you run this through GCC here, then there will--some optimization will
be happening here. You will no longer see this actual assignment here, so you will do
the--add immediately on those and one and two. So actually, by cross-compiling DEX instructions,
we are capable of generating fairly efficient code on the--on the target language. Now,
we have different set of stylesheets that do the very same for JavaScript. I mentioned
earlier that we also support the Palm Pre. So, we also have stylesheets that just simply
take this XML file that contains DEX instructions and--but maps then the DEX instructions sets
to JavaScript. So we can actually then also cross-compile an Android application over
to the iPhone--sorry, to the Palm Pre. So up to this point, all I've talked about was
the iPhone. I have not said anything about Android, right? So, I come now to the next
part of my presentation because now that we have added a Java layer over the iPhone, basically,
the next step will be to take care of the API mapping from Android over to the iPhone.
And if you look at it the way you write an Android application, of course, you first
of all use the Java language and you use an Android's specific API. So if you look on
the far left of this slide, you can see the Android application that runs on top of Android
is only making use of Android API. So whatever you see published as part of the Android SDK,
that is what the application uses. Now, what we do now is for the iPhone. So we have now
what we call the Android compatibility library. That is built upon a common device layer.
So basically within our compatibility library, we have a little extraction that actually
captures the essence of the smartphone, of the particularities of the smartphone and
those portions of Android that actually are generic to all smartphones. And I'll give
you some examples on the next few slides. But the way this works is now that the Android
compatibility library, it's also written in Java, of course, and it offers exactly the
same API as the Android API gives you. And all of it is cross-compiled to Objective-C.
So what you end up with is, again, a native application on the iPhone. Now, we do the
same thing over for the Palm Pre. So now except, of course, here, we cross-compile from Java
to JavaScript. And we have a different webOS layer that kind of adapts the common layer
API to the way things are handled on the Palm Pre. So, to give you some examples on how
this looks like, let me give you three different examples. So first example, and the first
example, I want to show you how we map an Android button over to an iPhone button. That's
a fairly simple example. So, if you look at an Android button, there's a class called
"Button", it lives in name space "android.widget" and in our implementation of our XMLVM framework.
What we do is we just basically--our implementation of the Android button is basically nothing
more than a wrapper around a "UIButton". And this "UIButton" that you see here in the first
line of this class Button is of course, now the Java wrapper that we have created for
the Cocoa Touch "UIButton". So, "UIButton" is the name of the class of the iPhone button.
And I have one little example here. It's a little bit more sophisticated. If you have
a button, of course, you want to install a click listener. Well, it's called a click
listener in Androids. So a click listener basically allows you to install a callback
whenever the user presses the button then the application would be notified via this
callback. And there is a method called "setOnClickListener" and you pass an interface reference called
"OnClickListener". And what we do in our implementation--now of course, the "UIButton" has something equivalent.
There is actually a method called "addTarget" and that serves conceptually the same purpose
as the "setOnClickListener" on the Android side. Now, of course, there is a different--there's
a different way of defining this delegate and on the Cocoa touch side, there is something
called "UIControlDelegate". So, basically, what we do here is we instantiate an anonymous
class, we overwrite the "raiseEvent". And then inside the implementation of "raiseEvent",
we basically sent the click event back to--on the Android side. So, what happens is if your
Android application is using an Android button, if you cross-compile it over the iPhone, what
you have on your native iPhone application is actually the "UIButton" as given to you
by the Cocoa Touch Library. If you push that "UIButton" on the iPhone, what will happen
is it will call "raiseEvent" of that "UIButton", that "raiseEvent" is then forwarding that
to the--reference "theListener.onClick" which basically is the callback into your Android
application. It's that when you--when you push the iPhone button, it--what will happen
is here now that you will actually end up calling the callback inside your Android application.
Now remember, all of that is being cross-compiled over to Objective-C, so also this little piece
of code here will be cross-compiled over to Objective-C. Okay, second example, in Android,
there is this infamous R-class. And the way you reference resources in Android is via
this R-class. So if you have, for example, an external image, if you have a PNG file.
The way you reference it is by making use of this R-class. Now, Android will automatically
create this R-class for you. So what I have on the top here is--what you will see is a
little bit of that generated code that Android generates for you. So imagine if you have
an image resource, a file called "ball.png", then what Android is going to do, it is going
to create a member, a field of static final class called "drawable", that itself is nested
inside of a public final class called R, the R-class that I just mentioned. And it will
assign a some kind of unique ID that ideally you will not see but when in your Android
application, when you actually reference that, the way you do that is--so, if for example,
if you want--if you have an image view and you want to actually load it with that "ball.png",
what you do on Android is basically you say "setImageResource" and then you just simply
reference this field of this public static final class by saying "R.drawable.ball" and
you can see as exactly a "R.drawable.ball". So, so far so good. That's the way the Android
people have decided to design the API to reference external resources. Now, how do we deal with
that during cross-compilation because when you think about it, this method here "setImageResource".
Actually, it takes an integer. So at runtime, in our compatibility library, what we would
see is we will see this awkward hex number here. So how do we actually map this back
now to a file called "ball.png"? Well, what we do is we make use of Java reflection. And
on the bottom of this slide here, I'll show you a little bit of code snippet that also
comes out of our Android compatibility library. And basically what happens here is, you know,
you can see that we--first of all, we're asking for a class object of this "R.drawable". So,
this dollar sign here is just the internal way for Java to represent nested classes on
byte code level. So, "R.drawable" is just the way you--you reference the nested "drawable"
class on byte code level. But first thing we do is we just retrieve the class object.
They were using Java reflection to get back all of the declared fields which is just basically
an array of field and this API comes from J2SE. There's nothing magic. That is just
standard Java stuff. And then you can iterate over the--all the fields and then we can retrieve
the name and we can retrieve the integer that is associated. So in particular, if we do
this little for-loop here, eventually, we will see--for the ID, we will see this hex
number here. And for field name, we will see "ball" as a string. And from that, we can
use them the string to create the file name and then to actually load the image resource.
So, we resolve that those resources, that runtime, by making use of Java reflection.
Now once again this piece of code on the bottom here is also cross-compiled over to Objective-C.
So, this gives you a bit of an idea that our cross-compiler is actually fairly powerful,
in the way that it can deal with this kind of Java reflection. Third example, Androids
has layout managers. So, Android--I mentioned before that Android is actually very well
equipped to deal with different types of device and one nice feature about Android is that
they have a way to describe these interfaces by a declare definitions. So, there are some
XML files that's--that basically describe a user interface. And on the bottom here,
I've just listed one of those XML files. So that is basically the schema of that XML file
on the bottom as defined by the Android team and what you see here is "LinearLayout" is
one of the many layout managers that you have in Android. And basically, it just defines
a vertical arrangement of different interface elements and what you can see is--here is
"LinearLayout" is it just--it has an "EditText", it has a "TextView", it has a "Button" and
then another "LinearLayout" that is--simply is wrapper around a little logo. And here
the screenshot on the right, you can actually see what it would look like in the preview
of Eclipse. Now, Android is Open Source. Android has an implementation, has Java code that
takes this XML file and basically parses this XML file and you will render this user interface.
So, what we have done now is--since we are an Open Source project, one that's simply
leveraged on the Open Source. So, actually we take the original Android source code of
the layout implementation, layout engine implementation, we also just simply cross-compile it over
to Objective-C. So at runtime, on the iPhone, we also copy these layout files over to the
iPhone. So, what happens at runtime on the iPhone that we read in this XML file and based
on this--this declarative description of user interface, we actually then start building
up the UI. So here again, we make use of Open Source and, you know, also get a sense of,
you know, what is all contained in our Android compatibility library. So it's partially,
the things we had to implement ourselves like the parsing of the R-class or the wrapper
for the widgets like Button but we also leverage the Android source codes in the sense of bits
and pieces that are device independent. So if you download XMLVM, there is a little application
that we have done now. We are tool developers. We are not game developers. So, don't expect
too much of that application. It's a little application called Xokoban. It's actually--the
gaming ideas is fairly old, you have to--it's a little strategy game where you have to push--well,
in this case, apple to target areas and so the little guy in the middle has to push these
apples to the white outlined target areas. And the way we have done this game is by implementing
it in Android. So, we implemented it in Java and it's only making use of Android API. So
the screenshot in the lower right is just simply a screenshot of that original native
Android version that we have implemented for the Xokoban game. Now when we cross-compile
that to the iPhone, it looks then like on the screenshot on the lower left side. Now
I mentioned before that Android is actually a very well-equipped with dealing in different
device capabilities. So, actually the way we have written the Xokoban game is that we
actually look at what is the screen solution on the device. Now, on the iPhone, at least
so far, it only has a certain resolution, 480 x 320. But the Nexus One for example,
on the Android side has a higher resolution. So actually our implementation of Xokoban
renders nicely even on high-resolution screens. So, this application has been pushed out to
the App Store, Apple App Store, and also the Android market. So if you have an Android
phone, if you have a Nexus One, or if you have an iPhone, you can actually go to your
respective markets and you can look for Xokoban. You can actually download the application.
Now I should make one little disclaimer here, because this screenshot is not exactly accurate.
And if you actually do download the application, you will notice that it's a little bit different
from the screenshot here and the difference is this little guy here in the middle, the
Android. You know, initially, we thought it's kind of like little inside gag, you know,
like Android pushing apples, you know. But Apple didn't like that, so actually, I had
an App--the App Reviewer calling me and he said, "Well, thou shall not have other Gods
besides me." Well, I didn't said that literally but what we basically he said is they don't
allow any competing--they don't allow references to compete in products. So, we had to replace
the Android logo with something else, so... So, now we have--the version that you can
actually download now is now--we're using the Linux penguin instead. And I guess for
Apple, that is not really a threat. So, they have they have accepted the tux as the guy.
Okay, that's going to be the last analogy between Apple and the Vatican I'm going to
make here. So to give you an example of this--a much more complicated and a much more levered
application, so we have been working with an Australian company called Funky Mobile
Games. And this company specializes on games from mobile platforms. And they have created
a version of a game "Stratego" for Android. So basically, they have a fairly elaborate
implementation of this game for Android. And they have used our tool, XMLVM, to cross-compile
their implementation of Legion over to the iPhone. Now, this application is making use
of quite a few Android's features, so they have like--they're making heavy use of activity
life cycle, they have sound, they have animation, they have custom dialogues, they use a lot
of declarative layouts for the UI and also use file I/O for the serialization of the
game state. They also have a fairly sophisticated AI engine, so they--actually, there's a--a
three time world champion of--of this game Stratego, who implemented this AI engine.
So, what I want to do now is actually I want to do another demo and I want to actually
show you this game of Legions side-by-side in both the Android emulator and also in the
cross-compiled version for the iPhone. So, let me first--so what I'm launching here now
is the Android version and I can turn up the sound just for a second here. I don't know
if the microphone picks it up but actually they have put a lot of effort into this game.
So they have hired a composer for the music. They have hired a design artist for the graphics.
So let me just turn off the sound here.
So let me now launch the iPhone version. And here, I'm in Xcode. So actually here, I have
loaded the cross-compiled Objective-C source code and I'm just launching the same application
now in the iPhone emulator. Whoa, it's a lot louder. And you can see that's already like,
you know we're mapping the sound API here. So let me just turn down the--or also turn
it off here to show you that's with the same thing. So, the first little thing that I want
to point out, if you actually look a side by side, the difference of the screenshot,
on the Android version, you have this exit button here which we don't have on the iPhone
version. Apple does not permit exiting of an application on the iPhone other than pushing
on the home button. So, actually we offer a way to the Android developer to find out
if your application is running on the Android or the iPhone. And depending on which platform
you run, you can actually then do things differently. So the Android version is actually also adding
this exit button which you cannot do on--well, you could do it on the iPhone but Apple would
not pass that through their App Review. So, there's like a help screen. So if you click
on help, then you have like some little help text here. You can go back to the main menu,
and let me just do that here on the, on the iPhone. Go back. Let me start a new game.
So here, I actually--I don't, I don't know the rules of this game. I know that you have
to place these elements here, so you have to pick like little soldiers, you have to
place them on the game board. That's like the first step of setting up the board. But
what you--then can also do is there's a feature called AutoFill where you can just simply
pick a random setup. Here's a little [INDISTINCT] that says that you can go back. You can say
start game. It scrolls down the game field, and now we can start the game or we can play
the game and do the same thing here on the iPhone. So again, you know, I have like these
hovering pieces with the zoom feature. I can place them manually if I want to. I can also
pick the, the AutoFill feature. I can go back. I say start game. Again, it scrolls down here.
Now, I can, I can play it by making moves. And again, I'm not--I don't really know how
to play this game, so I'll just do something here. I just got killed. I know that. And
then--it's thinking. It's doing its countermove, and let me make something similar here on
the, on this side. So let me also pick maybe this here. So it does animation. It scrolls
over here. Again, I got killed. And then, let me just surrender here because I really
don't know how to play this game. So, if you want to surrender, there's a custom dialog.
Yes. And then, we lose. It would tell you--it will actually open up all the fields. You
can actually uncover and see what you were playing against. Let me do the same thing
here on the iPhone. Surrender? Yes. And then, you'll see the same thing here. So, basically,
the--apart from this one little thing with the exit button, everything else is 100% pure
Android; 100% pure Java. All the across-compiled to Objective-C old met from Android API over
to the iPhone. So, going back to my presentation, of course, the question is then how far can
you push this? So, I already mentioned a few times now that Android certainly is very well-equipped
with dealing with the device heterogeneity. But what kind of applications can you actually
cross-compile? Well, games are certainly good candidates because games are not bound by
user interface guidelines. You know, like you have like your image buttons and you have
these custom dialogues, and a part from this exit button, actually, you can pretty much
do whatever you want. Now, if you have an application that is more based on widgets,
if you have a more business type application then, of course, things get a little bit more
difficult. And the reason is that, of course, every handset, every smartphone, has their
own proprietary sets of widgets. So, basically, if you ask me, so what did you do in that
case? Well, if you look at the model view control of paradigms, the way you should be
designing your application, what we can always cross-compile is the model. And we probably
can also cross-compile a fair amount of, of the controller. Now, the view you might want
to do a custom reimplementation based on the, on the particular smartphone. I mean the iPhone
is known for its very fancy widgets, so there's no point in trying to mimic an Android gooey
on, on the iPhone. So, probably a reasonable approach would be to actually then re-implement
the view portion of your application but making use of, of the smartphone specific widgets,
in that case, iPhone. Now, of course, I mentioned--as I've mentioned before, we have a Java obstruction
around the iPhone now, so which means that even though you may have to re-implement the,
the view for the iPhone, but at least, you can do so based in Java. And one thing we're
currently exploring is actually to, to mix iPhone and Android. So if you do a custom
implementation of, of a view for the iPhone, that basically, you can do--you can use an
iPhone widget but embed it into the Android framework. So you can take, for example, like
a picker, an "UIPicker", of the, the iPhone and you make use of it using the linear layout
from Android, so that that should be quite possible, and that's where we kind of going
to as a next step. So here's some a quick overview of the complete XMLVM tool-chain,
and there are lots of little boxes here. XMLVM as a research project began in the early 2000s
and we have many other use cases that we can do. Well, XMLVM, I already mentioned XMLVM
as a name is derived from the fact that we are--that we have a XML representation of
byte code and we're making use of XML technologies for doing cross-compilation. So, actually,
we do have a front end for .net. We have a front end for Ruby byte quote and we have
also have back ends for, for different languages. Now, all these different paths, you can take
through this tool chain of different maturity. Now, obviously, the smartphone portion is
what attracts attention now to XMLVM's that is the one that we're putting a lot of effort
in right now, and it is the most stable. So what I've shown so far right now is basically
this here. So we go from, from Java Class Files down to Objective-C, so that is what
I've shown you so far. Then we can also go down to JavaScript for the Palm Pre, but we
can also do different things. We can do something like, for example, we can go from .net, and
we can also go down to JavaScript. So it would be possible, for example, to write a C# application
and cross-compile over to JavaScript. Or we can also map .net byte codes to Java byte
code. Again, the various paths through these tool chain of different maturities. So, in
some cases, like this one in particular, I just have it on this slide, is more like a
proof of concept. But, at least, it's in, in the code repository, and I guess, the idea
of where the project get its name from. Well, I come to my last slides. And, basically,
you know, what I first told you about in the past hour is while we first have added the
Java layer on top of the iPhone and then we have created our own Android compatibility
library to be able to cross-compile Android applications over to the iPhone. And, I guess,
the benefits go without saying--well, first of all, we support the skill set of people.
I mean there are lot of Java programs is out there and you can tap into their skill set.
Android has a big momentum, so obviously, you know, supporting Android developers and
the only API that you have to know is Android. So we deal with all the device specific details
like iPhone and Palm Pre. So, overall, it reduces the development cost, and it reduces
the time to market. While since there's quite a bit of interest and--on XMLVM, at least,
the smartphone portion, we are in the process of building up commercial supports for XMLVM.
And well, what we plan to offer is a broad range of different services around mobile
applications because we have noticed that quite a few companies are in this kind of
situation where they are want to be present in this App space but they realized that they,
first of all, lacking the knowledge of the various platforms, but also they realized
the amount of effort they have to put in on targeting different devices. So, kind of to
fill that niche, and that is where we are in the process of building up the commercial
support. Well, that comes to the end of my presentation. I would like to thank you for
your attention. And if you have any questions, then I'll be happy to answer them.
>> I got a question here on VC. What is the support for things like OpenGL and things
like Compasses and some of the other hardware that's in Android but hasn't been in the iPhone,
at least, until recently. >> PUDER: Okay, so open GL. You're asking
about OpenGL, so OpenGL, we actually do map. There was one, one guy who map the OpenGL
API, so that's taken care of. The Compass API, we don't have yet. Now, I mentioned before
that Android actually is targeting different kinds of devices. So actually, Android allows
you to explore the capabilities of the device. So, if you write a good Android application,
it should cope with that situation where your application runs on the device that may not
have all the feature you need. How you'll react to it that is up to you. You may say,
well, sorry, I need--I really need the Compass and I--you don't have one on your device so
I cannot run, but we can certainly cross-compile that over to the iPhone. There's another question?
>> The choice to use XSLT for the code generation. Can you talk a little bit about that? I mean
it's a cute idea but, is it more than that? I mean, how would that compare to like an
ad-hoc just parsing the DEX code and...? >> PUDER: Well, so the question is a bit more
details about the stylesheets that we used. And, well, like I mentioned, the product XMLVM
itself came from the fact that I was trying to find ways of doing bicro manipulations
through XML technology. So, early I talked about .net. Actually, we do data flow analysis
using stylesheets. Maybe it's maybe pushing it a little bit too hard on the stylesheets
side but, I guess what attracts me for stylesheet is just the declarative way of expressing
code generation. So, basically, the little template I've shown you earlier for describing
the add-ins, it was like a very clean, and very nice, easy way to express, you know,
what should happen when you come across at an instruction on--in your byte code. Now,
of course, you can implement it in different ways, right? I mean there's no question that
you can also just implement it in Java, for example, and that will no doubt to be more
efficient than you if you do it stylesheets. But, remember, the stylesheets--running the
stylesheets happens at compilation time, not at run time.
>> Yeah >> PUDER: So, the inefficiency that we might
have in a slightly long of compilation times refers to compiling time not the run time.
The code is still efficient that we regenerate. >> Is it only slightly longer than the straightforward
parsing? >> PUDER: I didn't do--I mean, we don't have
any other reference implementations so I couldn't really tell you, you know, how we compare
in terms of benchmark. Yes, it is slower but by how much, I cannot tell you.
>> And one more question, the game you showed, the full game, so I'm assuming it didn't like
cross-compile out of the box. How much work was needed to change the original game that
they wrote for Android to actually pass the cross-compiler and work properly?
>> PUDER: Well, so your question was how much work that we have to put in to cross-compile
this Legions game? >> Yes.
>> PUDER: Or how changes were required in the Legions game itself?
>> Uh-hmm. >> PUDER: Well, first of all, we have to put
in quite a bit of effort to make--to push XML to this--to the point that it could cross-compile
Legions. >> Not counting the work on the cross-compile
itself? >> PUDER: Yes, but it's otherwise...
>> Yes. >> PUDER: ...the guys in Australia had not
to make any changes to their application. That is just the...
>> The exit like... >> PUDER: We just, we just go...
>> The exit button, get rid of that? I mean, it was...
>> PUDER: Well, of course, yeah. Well, that, I mean I--when I saw that, you know, I've
told them, okay, guys, you know, we cannot, we cannot cross-compile an exit button FL
won't allow that, so we introduced certain API. We told them about it but, to my recollection,
that was really the only that they had to change to make it work on the XMLVM. Everything
else, they were using from sounded API to activity life cycle, to the other layout managers.
We took exactly as is without requiring any kind of changes in their site.
>> They didn't need to tweak it like... >> PUDER: No, they didn't have to tweak it.
No. >> Very nice.
>> So what type of affordances do you have for debugging support?
>> PUDER: Yeah, debugging support is a good thing because the code that we regenerate,
of course, looks ugly, right? I mean if I can just go back here to x-quotes and I can
just simply click on one of the cross-compiled classes, you know, if you look at this year
that looks ugly, right? So, our philosophy is that Objective-C is kind of like the Assembly
language of smartphone development, right? I mean, ideally, you don't really want to
deal with it. But you are quite right when it comes to debugging, that--is a bit of a
problem because you cannot really correlate that back to the original Java source code.
Now, if you have a working Android application and it crashes on the iPhone, then the solution
is simple. There's a back end XMLVM. So, you know, I would suggest then, just post a back
report on XMLVM and we will fix it. Now, you--the debugging happens on the Android. And Android
gives you all the powerful debugging tools >> Right.
>> PUDER: So, if the iPhone version then behaved differently or crashes even, that just indicates
a back end in our tool chain, but, basically, we'll take care of that and you shield it
from that. So the good news is that the debugging, again, we leverage the Android STK.
>> The other question I had is what--how do you deal with--how do you let the developer
know when they're using a feature that's not really supported on iPhone? So, for example,
an Android like there's a text speech library that you can call from the APIs, which my
understanding is that API isn't quite available yet on iPhone, and...
>> PUDER: You would, you would notice during cross-compilation because it will be linking
our--you would try to link against something that is not present.
>> Okay. >> PUDER: We--so we don't have like, explicit
list, API list where you can look up and see which API we have covered. We should do that.
I do agree, but, at this point, if something is missing, something we do not support yet,
you would find out when you link because then if unresolved to externals.
>> Okay. Cool. Thanks. >> You mentioned support for the Palm Pre,
the Android and the iPhone. Is there any support for the BlackBerry?
>> PUDER: Well, of course, we get a lot of request. There's a request from--for Windows
mobile. There is request for BlackBerry. At this point, XMLVM does not support these as
back ends. But I think, you know, it's--we are in pretty good shape of very quickly targeting
also these platforms. So, the way we go about of this, you know, like if we have a client
who is interested in a certain feature then we certainly put that high in to-do list.
But at this point, our main focus is the iPhone. The second focus is the Palm Pre. And beyond
that, we just, you know, let customers decide on what they want.
>> Android has a SQL light database, I believe. It does the--this cross-compiling also support...?
>> PUDER: Okay. The question is SQL light. We don't have an API data for that at the
moment, but to my knowledge, the iPhone also has a SQL light database that you can also
access. >> So it's possible...
>> PUDER: So it would just be--it would be possible, yes. Again, the thing is, remember,
we don't have to have to re-implement the features, right? I mean when you talk about
Compass, when you talk about [INDISTINCT], when you talk about different features, usually
all these devices have these capabilities. So also a SQL light database is present on
all these devices, it's just creating this repose. So, actually, making this mapping
is not as complicated as it may sound, so we can very quickly--if there is a demand
for a certain feature, we can fill this gap if it's not present at the moment in XMLVM.
>> Android also has the OS itself. I think the framework or I guess it's--provides ways
to communicate between applications, like, you could register with the phone to be a
media player, and things like that. That's not possible.
>> PUDER: Yeah, well, okay. Of course, you know, if Android has this notion of activities,
and you can also call an activity from other application. So, for example, if you want
to select one of your contacts, you can call a self activity from another application.
Now, the way we plan to address that, and we haven't done it for contacts, but we have
done it actually for web views, we have implemented an activity that is--would be linked to your
application so that it looks like, on the iPhone, you're switching over to another application
when in fact, you're still within the same application that you have launched. So I think
in many cases, we can, we can deal with that, and so we actually, we do not do multitasking
which is not allowed on the iPhone anyways. But, we plan to implement special purpose
activities that kind of mimic what you have on the Android.
>> Do you kind of launch the dialer from--you link with the dialer, and when I click on
a button on my App, it'll actually show the dialer window that you linked with...
>> PUDER: Yes >> ...in my Apps?
>> PUDER: And then this special purpose activity then would mimic whatever you see on the...
>> Yeah. >> PUDER: ...on the Android, but it will also,
of course, make use of--it will make use of iPhone's specific features. Another questions?
Well, if not, then, again, thank you for coming today, and thanks for listening.