Dartisans ep. 11 - Compiling Dart to JavaScript

SETH LADD: Welcome everyone to Episode 11.
And today, we are going to talk to engineers from the
dart2js team.
And so stick around, and you could learn how we get Dart
code to run across modern browsers.
[INTRO MUSIC]

JJ BEHRENS: I'm JJ Behrens.
This is Seth Ladd.
SETH LADD: Hello.
First, the news.
This all comes thanks to Chris Buckett's dartwatch blog.
So thank you for collecting all the news.
First off, Mike Eberhart has created Dart Squid, his UI SVG
components.
Now, have you worked with SVG?
JJ BEHRENS: No, actually I haven't.
For some reason, I've never gotten a chance do that.
SETH LADD: I think it's underappreciated technology.
And I look forward to more deployments of SVG, and
especially with Dart.
So check out Mike's library.
JJ BEHRENS: The thing I don't like about SVG is I'm not a
very attractive guy.
And so what I'd like is taking a picture of myself as a GIF,
and then blowing it up, I'm kind of fuzzy.
But in SVG, it's like--
SETH LADD: It is vector graphics.
JJ BEHRENS: Yeah.
It's like you could see everything.
There's no hiding behind the fuzz.
SETH LADD: Simon Pai, from the Dart
framework builders Rikulo--
and you can remember them from a previous episode--
has blogged about a rotatable map prototype.
So it's really cool to see another UI
framework built in Dart.
And these guys are friends of the project.
And it's fun to track their project.
Florian Loitsch, one of the members of the dart2js team--
JJ BEHRENS: We'll hear for him in a minute.
SETH LADD: Yes, will be speaking on JavaScript as a
compilation target at JSConf EU.
Now, this is really cool.
It's a heavy JavaScript conference.
But we're going to talk about how we get Dart
to JavaScript there.
JJ BEHRENS: Yeah, that turns out to be a lot more complex
than I thought it would be.
SETH LADD: And with the Pub, the package manager for Dart,
getting released very soon-- in fact
it's in the SDK today--
the server components are coming online shortly.
And you see more documentation posted about how to create a
package, what does the versioning system look like,
and how to declare your dependencies.
In preparation, the Dart M1 language changes.
Now this is cool.
The editor will help you quick fix or clean-up your--
it sounds weird, but legacy dart code.
That is the code before the M1 line in the sand of the
release, if you will.
And so it will help you update the syntax for things like the
try catch blocks, interfaces to classes, and more.
And so Definitely try this out.
And as more implementations of the new language features come
online, the new editor clean-ups will get even more
and more useful.
JJ BEHRENS: Dude, this is frickin' magical.
For a long time, I was super jealous over Go, because Go
had this nice thing where it could automatically upgrade
your code to new APIs.
But now we have it directly in the editor.
I'm super psyched about this.
SETH LADD: Yeah, this is awesome.
So try out that feature and let us know how it works.
So John Evans, one of the early adopters of Dart and the
author of the Buckshot UI framework, has been blogging
about his experience using the Mirrors API.
Now, mirrors are awesome because it's our way to do
reflection, things like tell me more about the class I'm
operating with, and I think in the future, do things like
construct a class on the fly.
Finally, for more news, we have a 83% more awesome
api.dartlang.org site now, with a "find as you type"
search feature.
Now, have you tried this?
JJ BEHRENS: Yeah, briefly.
But I'm really wondering, how'd you
come up with the 83%?
SETH LADD: It's very complicated.
We had to use 6,000 cores in Compute Engine, wrote it all
with Dart, but it is verifiable.
And it's really nice because we have a lot of classes and
interfaces that use, for instance, the word "element".
And sometimes you want to just find the superclass element.
Now you go right up to the search box type "element," and
it comes up in a nice ranking order.
And so much easier now to browse around if you just want
to know what does our API provide.
So definitely try that out.
JJ BEHRENS: It's funny, if you're a language guy like I
am, you kind of spend all your time
thinking about the language.
But actually, a lot of those other things actually really
help out your productivity.
Because spending a lot of that time searching through docs
instead of having something that immediately takes to what
you want could really increase your productivity.
So I'm excited about that.
SETH LADD: Yeah, yeah.
Great.
Well, that's what we have for the news.
So now let's invite our guests.
JJ BEHRENS: Wait, hang on.
I'm going to do it.
You ready?
SETH LADD: One, two, three.
[SNAPS FINGERS]
SETH LADD: Ta-dah!
JJ BEHRENS: Hey, welcome to the show, guys.
FLORIAN LOITSCH: Thanks.
SETH LADD: How was that?
Your first magical teleportation.
KASPER LUND: Not the first, no.
SETH LADD: No, not the first, right.
PETER VAN DER AHE: It's my first.
It's more easier than I expected.
JJ BEHRENS: Yeah.
So can you say something cool in Danish?
KASPER LUND: [SPEAKING DANISH].
JJ BEHRENS: I can't tell if that's cool or not.
KASPER LUND: [SPEAKING DANISH].

JJ BEHRENS: Yeah, well, that's obvious.
SETH LADD: Bring in some good YouTube comments from that.
So most importantly, how do you pronounce
[SPEAKING DANISH]?
PETER VAN DER AHE: [SPEAKING DANISH].
SETH LADD: [SPEAKING DANISH].
KASPER LUND: That was pretty close.
JJ BEHRENS: You guys want to introduce yourselves?
FLORIAN LOITSCH: Yes, I'm Florian Loitsch.
And, I've been working on Dart since the beginning.
My background is basically scheme and compilers.
I wrote scheme to JavaScript compiler.
And then I wrote JavaScript to scheme compiler.
And then I joined the V8 team.
And then eventually, I transitioned to Dart.
And now I'm working on a Dart to JavaScript compiler.
So JavaScripts--
SETH LADD: It always comes back to JavaScript.
FLORIAN LOITSCH: --and Darts, dynamic languages, scheme.
That's my background.
JJ BEHRENS: Awesome.
JJ BEHRENS: And I'm Kasper Lund.
I worked on Dart since the beginning, too.
Now, I'm working on dart2js, compiling Dart to efficient
JavaScript code.
Before that, I spent some years on optimizing JavaScript
in Chrome and the V8 engines.
I had a lot of work in trying to make
JavaScript run really quickly.
So now it's nice to be able to make good use of that by
compiling to it as a target language.
PETER VON DER AHE: My name is Peter von der Ahe.
And I work on dart2js.
I mostly focus on the front end, that is, passing symbol
resolution and stuff like that-- type checking,
inferring types, perhaps.
And then before joining Google, I had worked on Java
Caesar, the compiler.
And I've also worked on Gilad Bracha's Newspeak project.
JJ BEHRENS: Wow.
SETH LADD: We should do a shout out to the rest of the
dart2js team, who we only have this big of a couch, but we
have a really awesome team.
We want to thank everyone for all their work.
And I should probably say, we have an external contributor
working on dart2js.
Is that right?
KASPER LUND: That is true.
SETH LADD: It's our first installed contributor.
PETER VON DER AHE: After submitting a few CLs on his
behalf, we asked him if he wanted to be a full-scale
committer on the project.
JJ BEHRENS: That's pretty amazing.
KASPER LUND: That's nice.
More resources.
JJ BEHRENS: I also think that really points out the open
source nature of Dart.
SETH LADD: So I'm really interested in the design of
the language and how it's been impacted by what has always
been a core belief of ours that it has to compile to
fast, sane, logical JavaScript.
Can we talk a little bit about how that's impacted?
KASPER LUND: Yeah, from the get-go, we knew that being
able to run on any browser was really important for Dart to
be interesting to people that wanted to develop big, large
web apps for the real world.
So we looked at the constraints imposed by using
JavaScript as a target language.
And some of the things that we probably wanted to try out in
the language were really hard to do when compiled to
JavaScript.
A few of them are things like non-local returns, where you
can return from an inner function.
And you actually end up returning, not from the
function itself, but from the enclosing functions instead,
which is a really cool feature.
It's just hard to implement in JavaScript efficiently because
it ends up being something with their own exceptions.
So that's something we dropped and said let's
not do that for now.
It's too complicated to make that efficient.
So let's keep it closer to what you can do in JavaScript.
There are a few other things that we also decided not to
push too hard on.
On the syntax front, we tried to stay reasonably true to
JavaScript so it would look familiar,
feel familiar to people.
I think that's very important for adoption.
FLORIAN LOITSCH: I think in Java, when we had the choice
when we weren't sure and there were several valid choices, we
picked the one from JavaScript just to make it comfortable
and to make it familiar to JavaScript developers.
KASPER LUND: You could say that the async programming
style that we use in Dart is also inherited from
JavaScript.
We're not ruling out allowing users to tell us to trust the
type of notations if they really need that for
deployment.
It's just putting that in too early will essentially remove
all pressure from us to deliver something that's
really, really nice without those slightly hacky
optimizations, right?
So we like to have that pressure put on us to deliver
something that's really fast without unsafe optimizations.
SETH LADD: I think it's also a workflow issue.
The way we envision Dart being deployed a lot is it's a very
scalable language.
So you can start with just some functions and a small
prototype, and you don't really have
types in play here.
And we want people to start that way.
And so we don't want people to necessarily type everything at
the very beginning.
Of course, you can if you'd like, but you should have that
option to scale up to very type systems starting from a
very small R&D project, if you will.
JJ BEHRENS: Definitely.
SETH LADD: Before we take the questions, I want to actually
dive in a little bit to what happens in
dart2js, for instance.
So I think this is a good point for our viewers, who
might come from, say, a CoffeeScript background, or
who had this idea that, OK, I can compile it in one language
or convert one language to another.
But in dart2js, I think we have different semantics and
that governs non-straightforward way.
JJ BEHRENS: It's not a syntax transformation.
SETH LADD: That's what I'm trying to say.
Thank you.
JJ BEHRENS: It's even a semantic transformation.
SETH LADD: Yeah, so can you talk about that?
For instance, what happens when I compile and add "int a
comma int b" into JavaScript?
KASPER LUND: So Peter was sort of saying that the biggest
problem is really that you cannot just compile a Dart
plus to a JavaScript plus.
And that's exactly right.
And so for method calls, like calling a "foo" a "bar" on
some objects, it's sort of a pretty direct translation.
We have some slightly more advanced ways of passing
arguments, which needs to be dealt with.
But other than that, we're just using JavaScript function
calls to implement Dart function calls essentially.
But on the primitive values where we use JavaScript
numbers and stuff, it's a bit more complicated.
Because in Dart, you can provide a user-defined
implementation of an operator like plus, and it needs to
work in the right way.
Whereas in JavaScript, there is really no way of defining a
user-defined plus operator.
You can just use the one that's built in.
And if your optics have the right value-off method, they
might participate in this kind of built-in plus operation.
So Dart is more flexible there than JavaScript.
But it also means that there's sort of a semantic gap between
Dart plus and JavaScript plus.
In itself, it's pretty easy to implement Dart plus.
It's just very slow if you compile to JavaScript and just
essentially turn a plus into a method call.
So we go through I would say great lengths to try to figure
out when it's safe to compile a Dart plus into
a JavaScript plus.
And it's only safe under certain assumptions.
It's only safe when you know you're operating
on JavaScript numbers.
Otherwise, you're not getting the right semantics.
For instance, if you end up operating on JavaScript
strings instead, you'll get string concatenation out of
it, because that's what JavaScript does for plus, and
that's not the Dart way.
So that's one of the big challenges we have is just
pushing enough type information around the system
to know when we can safely translate a plus to a plus.
FLORIAN LOITSCH: Then we also have speculative
optimizations.
So, in particular, for arrays or array accesses, we don't
want to go through helper methods to do that.
So what we do is if we see that this might be a loop that
works with arrays, we do the check.
If it's a JavaScript array before the loop, we check if
the index is an integer.
And usually the index is just incremented to the loop.
So once we know it's an integer, it
will stay an integer.
So we do that before the loop.
And in the loop we have a nice efficient access.
And in case it's not an array or it's not an integer, we
have a bailout method.
So we jump to another method, and there we do the slow
accesses where we go through helper methods to make sure
that if the array is actually--
if the indexed object is actually not on array, then we
go through the methods of it and make sure that everything
works as expected.
KASPER LUND: And again, that's because indexing into lists
and collections in Dart is a user-definable operator.
So you can write your own collection type that allows
you to index into it.
That's not possible in something
like JavaScript either.
So we have to make use of the JavaScript primitive indexing
operation when we can, when we know it's safe, or when we can
generate essentially a two-code pass where it's safe
in one and maybe faster.
Or we just do this bailout version that will do the right
thing, but be slower.
So if you ever look at the generated JavaScript, the
output from the dart2js compiler and find these
slightly weird bailout methods, that's the reason.
It's because we're doing speculative optimizations
where we're not guaranteed at compile time
that this will hold.
So we have to generate two versions, essentially.
One thing that many people probably don't know is that,
at least on certain JavaScript VMs, at least the one's I'm
intimately familiar with which is mostly V8, there's actually
a cost associated with passing the wrong number of arguments.
Passing too few is costly.
Passing too many is costly, too.
It's not cost in the sense that your program will be
super, super slow if you get it wrong.
But just to make essentially the sequences and instructions
you have to execute to call something and return efficient
in the common case, it's heavily optimized for the case
where you pass exactly the right number of arguments.
So there's a good reason to do that if you care about
performance, especially for the hot loops in your code.
JJ BEHRENS: "Will dart2js be able to output compressed
minified JavaScript like the Google Closure compiler in
advanced mode?
If it can compress properties, it also has to create a
property mapping text file that could be used in web
services JavaScript serializers." Well, we get the
first half of this question a lot, don't we, Seth?
SETH LADD: We do.
And you could just see a patch that landed very recently--
dash, dash, minify.
Tell us what happens when you use that new
command-line argument?
KASPER LUND: So I can take responsibility for the
command-line argument.
And Florian can take responsibility for what
happens when you use it.
[LAUGHTER]
KASPER LUND: The Minify option is the answer to this
question, essentially.
Yes, we will be providing a way of outputting minified
compressed JavaScript output.
And we will make good use of the fact that we understand
the program structure.
And we'll use that to generate really compact output.
You could tell people what happens if you use it.
FLORIAN LOITSCH: So currently, when you use dash, dash,
minify, the only thing that happens is that they are going
to old unnecessary wide spacing, and the produced
methods is going to be removed.
So it's nothing much, but in our benchmarks we were able to
get down by 15%, maybe?
KASPER LUND: Yeah.
FLORIAN LOITSCH: And that's just by removing
all the wide spacing.
In future versions, yes, we want to have a nice minifier,
and for us, it's definitely easier than for a tool that
runs after on our generated code.
So we will definitely have something that will minify the
names and the properties and the cast names and everything.
And when we do that, we will provide that
mapping text file.
That shouldn't be difficult.
And there's no reason not to do it.
SETH LADD: In fact, today we're spitting out
a source map file.
We're not complete with that feature yet.
But it is going to be part of this
compilation and build step.
And it'll allow you to take minified or unminified
JavaScript output, and through the DevTools of Chrome, map it
back to the original Dart code.
And it's a really cool development workflow.
JJ BEHRENS: So this question isn't about dart2js, but it
seems to be really popular.
"Is there a public roadmap as to when we can expect Dart to
be ready for serious deployment?
I'm thinking about trying out larger projects and riskily
future proofing them with Dart."
KASPER LUND: I don't think we have a public roadmap for when
we're ready for that.
Right now, we're very much focused on delivering this
first milestone release, where we'll get all these new
language features implemented across the board so you can
start relying on them in the editor and
dart2js and on the VM.
So you can really start making use of these new things.
JJ BEHRENS: Yeah, it's definitely been my experience
that Dart is already a really, really good tool for building
web stuff and is a really productive tool.
But you have to understand that you have to
move with the times.
We're not trying to be backwards compatible ever
since last October.
We're always moving forward.
And as long as you're willing to move forward with us, then
I think Dart's a really productive tool.
SETH LADD: And I should say that Dart is
an open source project.
You can follow along on our mailing list.
You can follow along on bug tracker.
We publish updates from the team.
And so I would say certainly give it a shot now.
And if there's something specific you're looking for,
let us know on the mailing list.
And if you're considering it for a big project, we would
love to know what you're doing and what your priorities are.
JJ BEHRENS: So "as a web language, Dart needs to
interact with existing web applications which use
JavaScript.
How does the dart2js compiler enable interaction with legacy
JavaScript code like jQuery?" This is
another popular question.
KASPER LUND: It's actually a very good question.
So it's actually a very difficult question.
Because whatever we do, we want to make sure that you
have the same experience when running dart2js and running on
the Dart VM so that code you write today
will also run tomorrow.
So right now, the JavaScript interrupt that we have is
fairly difficult to use.
We have some examples on how you can build simple things
that interact with, for instance, Google Maps and
things like that.
But it's somewhat painful.
And we're spending some time on trying to figure out if we
can make it easier to use in a way that still works both on
the Dartium, Dart VM-enabled Chromium
builds, and on dart2js.
SETH LADD: So we should talk about one of things that makes
this a really tough problem for us, which is now that you
have multiple run times in the browser, you run into what we
call a Distributed GC situation.
JJ BEHRENS: Garbage Collection, Distributed
Garbage Collection problems.
That's a challenge.
KASPER LUND: It's a big challenge, right?
So you essentially have at least three different memory
management systems in the game.
You have the system that governs all the DOM notes in
your web page.
You have the JavaScript memory manager, and you have the Dart
memory manager.
And they need to coordinate things.
And that may sound pretty easy, but in reality, you want
the garbage collectors in those systems to be really,
really efficient, incremental, and give you essentially no
pause times and just great throughput.
So combining that with a system that tracks references
between them is just very, very difficult.
So it's something that we're trying to wrap our heads
around how to approach and how to solve.
But it's certainly not an easy task to get through that.
JJ BEHRENS: "Can the work on type propagation improve the
code size and speed even more in the future?
We still see pretty huge chunks of bailout code and
type agnostic code generated, which seems not necessary in
many cases/" So we kind of touched on this.
FLORIAN LOITSCH: That's an easy one.
That's yes.
Yes.
PETER VON DER AHE: We have only scratched
the surface, I think.
KASPER LUND: Yeah.
So over the last month or two, we've actually seen a lot of
progress there.
And I think this question refers to the fact that it is
improving a lot, but we still in certain cases see a lot of
things that could get even better.
And that's nice for us because that will keep
us busy for a while.
But it's certainly something we focus a lot on, like
getting very good performance out of the generated
JavaScript code through type propagations, type
inferencing.
There's a lot of work going on in that area.
FLORIAN LOITSCH: So the current status is basically we
have been really very good in local type inference.
So if there's a variable and we know the type then we
propagate this type and use it even for speculative
optimizations and things like that.
And recently, we started to use types on arguments and
return values and sometimes on fields if they have been
initialized only in specific occasions, or not too often,
and we can track it.
But there is work on a global type inference tool that will
help to get much, much more type information.
At the moment, we have type information
produced by the code.
JJ BEHRENS: So "what features of the Mirrors API do you
think will make it into dart2js sooner than later?"
PETER VON DER AHE: Introspection.
That's basically looking at the program structure, but not
modifying it.
JJ BEHRENS: I've heard a lot of people ask for this sort of
thing, because frameworks like AngularJS rely on reflection
as a core feature of how they work.
And so I know that a lot of people were interested in
seeing reflection in Dart as well.
PETER VON DER AHE: Yeah.

My take on that is that, yeah, you can write a lot of
frameworks like that, and you're probably gonna end up
feeling really clever when you get it to work.
But it's probably also frustrating when it's not
working for you because there's a whole bunch of
things going on, right?
First you have--
well, in AngularJS, I don't know if they have--
that's really a JavaScript framework, right?
It's not a language.
So there's no compile time step there.
But we already have a compile time step.
So we should sort of take advantage of that, and then
check your program there, and then generate
some efficient code.
JJ BEHRENS: Check for static warnings.
PETER VON DER AHE: Yeah, well, we have a compile time step.
So we should take advantage of that and use that.
But there's also--
for us to put in all the information that is required
to do introspection or modifying the program, then we
also have to follow those convent-- so that when we
extend the program, the code we have already generated has
to be compatible with how it's extended.
So all of a sudden, reflection can have an impact on how we
generate code.
So the first cut is prob--
that's why we have designed the mirror system as it is.
It's basically--
it's a capability that you request.
In this case, it would be by importing a library.
If you import this library, that may have to change how we
generate code.
FLORIAN LOITSCH: So just to give some examples of what
changes would be needed.
If we mangled the feed name or if we mangled the cast names,
well, then we would need to ship some information to be
able to get back from the mangled name to
the original name.
Similarly, if you are actually allowed to change, if you add
a new codes to your reflection, we cannot tree
shake anymore because you might call a method that
wasn't called before.
KASPER LUND: A request for feedback from anybody watching
this that's interested in introspection.
And we love to get use case on what kind of things you would
like to see there and which parts of the introspection
APIs are you using.
And then we can start to think about whether or not it's
realistic to get that rolled into dart2js in the
not-too-distant future.
JJ BEHRENS: So "will it be possible to write a new
library in Dart that could be used either by Dart or plain
JavaScript?"
KASPER LUND: So it's certainly something we
would like to see happen.
For us, we think that it's--
at least for certain things, components you want to write,
it's more efficient to write them in Dart than it is in
JavaScript.
So if we could allow you to do that for certain components in
your big project and just use them for other JavaScript
things, it would be kind of neat.
It's something we're looking at doing for the web
components work we're doing.
It would be very, very cool if you could take a web component
that has behavior written in Dart, compile it to
JavaScript, and use it from any web page.
It's certainly something we'd like to see.
So it's a great question, and hopefully, we'll be able to
answer it with a, yes, let's do it, very soon.
JJ BEHRENS: And then last one.
"If GWT can compile JavaScript to run on Android browsers,
then why can't Dart?" I think I've been hearing some
complaints that the JavaScript doesn't work
on the Android browser.
FLORIAN LOITSCH: Committed today.
JJ BEHRENS: Oh, committed.
FLORIAN LOITSCH: I just committed a fix for the
obvious syntax problem.
I'm expecting to hear other bugs that show me other maybe
different problems.
But it should work.
At least the syntax that was really making it impossible to
run on Android processes has been fixed.
KASPER LUND: And actually that's another place where we
got a nice patch from an external contributor.
That was kind of nice.
JJ BEHRENS: I would also say if you could run Chrome for
your phone, it's a nice experience.
FLORIAN LOITSCH: But to actually answer the question,
why can GWT compile, and why can't we?
Because they spent a huge amount of resources doing it.
KASPER LUND: It's always a matter of prioritization.
So it's great when we get feedback from people that say
I would really use this if it worked on Android browsers.
It's something we can use.
And we can think about whether or not it makes sense to put
in the extra effort to make it run.
JJ BEHRENS: When I compile my Dart code to JavaScript using
dart2js, it doesn't really work in Lynx,
my text-based browser.
KASPER LUND: I don't think you've tried it.
It actually works just fine.
JJ BEHRENS: Does it, really?
KASPER LUND: No.
[LAUGHTER]
JJ BEHRENS: So thank you guys for all your time today.
It's been a great show.
And I think we will see you guys back when Seth and I
return to California.
And we'll do our next episode in two weeks.
SETH LADD: Absolutely.
And thanks for watching Dartisans.
You can find the Dart project at dartlang.org.
Do join our mailing list at dartlang.org/mailing-list.
And we have bug tracker at dartbug.com.
Lots of good ways to get in touch with us, and we're
always looking for feedback.
I mean, we certainly prioritize things based on
what we hear.
We've taken external patches.
It's a really cool open source project.
And so please do let us know what you're using Dart for and
what you need it to do.
JJ BEHRENS: Goodbye from [SPEAKING DANISH].
[ALL SPEAKING DANISH]
[MUSIC PLAYING]