Selenium: The in-browser acceptance testing tool


Uploaded by Google on 23.07.2007

Transcript:

JASON HUGGINS: All right.
So Selenium.
I got a couple of name drops yesterday in various sessions,
so I was staying on the down low yesterday.
I poked my head out.
Anyway, yes, so I'm the original author, but now as
time goes on, my contribution to the project gets less and
less as more people come in and refactor ugly code that I
may have written in the past, or build add-ons,
something like that.
So just a show of hands, I'm just a little curious.
How many people are aware of Selenium
or have used Selenium?
I thought it was about half.
It's a little bit more than half, so maybe I don't need to
do so much tutorial stuff.
I'll do some of that stuff anyway, because I have to wave
hi to my mom on the video.
So let me just get to the slides, and then get past
that, and then go to the actual stuff.
So there is a problem.
The problem is complexity.
And this was my complex problem that led to the
development of Selenium.
For our in-house time expense application that I wrote with
a team of other ThoughtWorkers.
At ThoughtWorks they're a very unruly bunch.
You can't just say, use Internet Explorer.
We give them Dells, and you say, use your Dell, and they
go ahead and buy Macs.
So we have to support Safari and Firefox and Internet
Explorer, just from the user base.
Then also from a development mindset, we would have
developers who would only do stuff in Firefox, and check it
in, and then the next week we find out it's broken in IE,
and vice versa.
And so this is a very complex thing.
And so when we created Selenium, this was a year
before AJAX.
This was back in the days when JavaScript was bad.
We had a lot of Java developers on this sort of
Java good, JavaScript bad.
And so Selenium actually, it took a
couple of aborted attempts.
Finally I was able to convince somebody, no, no, no,
JavaScript is good.
And so that's why Selenium, I think, was in this weird right
place at the right time.
Because it grappled this problem, which only now sees
JavaScript as good, people care about this stuff.
And then also, these other things at the bottom.
Later on, people didn't really like the original format of
Selenium, so they want to be able to write their tests in
Groovy and Python and Perl and Java and C#, be able to use it
on any different kind of framework, and all these
different kinds of things.
So we tried to distill down all this complexity into a
relatively simple tool, but nothing is ever that simple.
So just to repeat, how do you test your web app on all
platforms, browsers, web frameworks, and regardless of
implementation language or the language you want to write
your tests in?
Sometimes they're the same.
Sometimes they're not.
So the solution, of course, automate it.
That's why we're all here.
And how?
With Selenium, of course, with a couple of nifty bits that
I'll show you.
So Selenium is actually a minor player in all the stuff
I'll be showing you today.
So a little bit of the context, the ingredients I'm
going to show you, I had two converging ideas.
But the main one is I wanted to see if you could use
Selenium to build--
oh, let me show it.
Let me ask you a question.
Has anyone heard of these sites called like
browsercam.com, sitevista.com, browsershots.com?
Anybody?
Rings a bell?
OK.
So you give these sites a URL, and they give you a screenshot
back of that page.
And you actually can select from a grid, like, oh, I want
to see Firefox on Linux and Safari, whatever, Mosaic on
Windows 95 or some wacky thing.
Essentially I'm sitting back there, I saw that, and I was
like, oh, wow.
That would be cool.
Maybe if you gave it a Selenium test, you can not
only just take one screenshot of one URL.
You could say, take a screenshot, oh, now go click
here, type that, then take another screenshot.
Or maybe not do that, just record a movie.
And then, taking that idea towards its absurd conclusion,
there's this convergence of idea in the sense that writing
tests is very boring.
There's very little payoff.
But this tangent I'm trying to bring together, all the cool
kids are doing screencasts for their projects.
I was wowed like everybody else when I first saw the
movie for Ruby on Rails.
Granted, I don't think you could probably script that all
through Selenium.
But I had this idea that kind of one step down the road of
literate functional tests for story-tests,
actually making your--
I guess in this case, your Selenium tests executable
documentation.
So your documentation that you deliver to the client, if you
make it easy to read enough, that you don't have to spend
another week making screencasts, actually, every
time you check in, you get a compiled screencast in Safari,
Firefox, and IE.
So that's what I tried and mostly got working for this
talk, so these are all the ingredients to this idea that
I have somewhat working on my computer.
So Apple MacBook.
Interesting story about this.
I didn't own one until last month.
My top proposal was to--
well, this other converging idea, now that Macs are
Intel-based, you can run all these different
platforms on them.
And I had originally that problem of needing to test
Safari, Firefox, and Internet Explorer.
Well, a Mac mini or a MacBook actually
would solve that problem.
So yes, I didn't own a MacBook, so I got accepted to
this talk saying I'm going to be using a Mac, so this is
basically the subtext.
This is the way I got my wife to agree to let me buy a Mac.
I did try to actually do all this with a stock MacBook, but
it was 512 megs of RAM.
The second put on one extra virtual machine,
everything was crazy.
So two gigs of RAM.
If anyone wants to repeat this at home, go out and buy RAM,
because it doesn't come with enough.
Well, for all this crazy stuff I want to do.
So the next thing, Parallels Desktop virtualization
software, and I guess VMware is close on their trail.
Now granted, if you didn't care about Macs and Safari,
you could do all this stuff just using VMware.
Actually, that's my one concern about this, is that I
like being able to use-- it would be nice to develop here,
and then to take a virtual machine and
deploy that on the server.
It's not quite easy, because Parallels
has their own standard.
They're not quite interoperable between all the
different VM formats yet.
So that's one little side note there.
One other little bit in my bag of tricks, just this summer a
couple other ThoughtWorkers based in the UK put together a
Knoppix-based distribution called Buildix.
And it's a tightly integrated--
it's not really supposed to compete with all the other big
Linux distributions, but it's nice, tightly integrated
CruiseControl, Trac, and Subversion.
And me, being the JavaScript and Python guy, I'm deathly
afraid of Java.
So I've actually avoided CruiseControl for all these
years, because I thought, oh, that's a Java thing.
And then there's Buildbot, I guess the Python community,
but Buildbot looks ugly.
So I'm just kind of stuck.
CruiseControl actually does look OK.
But now that they put that in a virtual machine, I don't
care that it's anything.
I have to edit a couple of Ant files, which are bearable, but
all I have to do is just install into a virtual
machine, and use it, and run, and it works.
And it's nice.
So that's actually the radar, traffic control of this, the
brains in this little thingamajig.
Also, as far as Selenium, there's
many aspects of Selenium.
It confuses people.
But I'll be showing you a little bit, the Selenium IDE
for Firefox that's the record/playback tool, which
may or may not be useful or good, but it actually does
record/playback.
We did notice a huge spike in interest in the project, even
though, some testing experts might say
record/playback is evil, bad.
But you have to go where your users are.
And our users start with record/playback.
So at least it can get them hooked, and they you can lead
them on the trail of writing better tests.
And then also, we have this kind of advanced version,
Selenium Remote Control Server.
And that lets you--
it's a Java proxy server.
Again, I didn't write it, but the cool thing about it is
that it's a proxy.
So you can write your tests in Perl--
you guys all know this, but--
Perl, Python, Ruby, Java, C#.
It proxies the requests over HTTP, gets and posts, and then
it knows how to launch various browsers, and configure them
so pop-ups don't show up, and things like that.
I'll just show you, briefly, Remote Control.
And then, of course, Python.
I don't know why it needed, really, its own slide.
It was very cool for this little automated screencast
thing, pyvnc2swf is very strong sauce, I guess is the
phrase, and ElementTree.
It was stupid simple to take an RSS feed and get the parts
that I needed.
So I'll just show you a little bit of my setup here.
Then I'll actually go back to showing you Firefox IDE, some
of the bigger parts of the toolkit.

So I've got this setup.
This is a plug-in called VirtueDesktops.
It lets me have six virtual screens.
I've got four virtual machines running.
So I've got two just Windows, for my Mac stuff.
But I've got Buildix running in its own screen.
Then I have Django, which is running on an Ubuntu server.
Now, this is different than I think the way
Buildix has been described.
I actually set up my own test server, with the Django web
framework. and then I had SSH connections back and forth
between Buildix and Django.
So this is interesting, because then I actually can
take that virtual machine, it's very uncorrupted by the
Buildix parts, and I could actually put that on a
developer, give that to a developer, and say, here you
go, Here's your developer playground.
Or maybe not using Parallels, but VMware, this could then--
I'm deploying to this virtual image, but if it was marked
good, I could probably, a simple matter of programming,
perhaps use that as my [? production ?] box in a VM
grid kind of a thing.
So I wanted to separate out the build software from the
actual test framework server.
And then I've got two desktop operating systems. I've got
Ubuntu Desktop and I've got Windows XP.
And also, it's not really down here, but behind the scenes,
I've got a second concurrent user of Mac OS.
It'll be doing the screencasts in that user account.
So it won't be taking up much screen here.
So I'll just do the quick fly-by, because
this thing is so cool.
That's Buildix, really it's just running headless, so it's
really nothing.
Same with Django, nothing.
Interesting.
This is interesting.
So I've got full Ubuntu desktop.

I've got on Ubuntu, and then on the Mac user--
hmm.

These permanent loops going on, listening for check ins on
the Buildix box.
And if it doesn't work, then my demo's not
going to be so great.

Yes.
Well, I've got some pre-canned movies in there, so I think
I'll show you those instead.
I've got Windows and there's not much to say about that.
So that's the setup.
I'll briefly just show you what Selenium looks like, for
those who don't know.

So it demos well.

So I'll just click All, Run.
And this is it just doing its thing.
This is the original version that I can lay claim to, like
the layout.
The whole thing was I liked Fit, but I didn't like the
fact that it just did stuff behind the scenes, and then
said, oh, I did it all, and it worked.
Like, really?
So originally I actually emailed Ward Cunningham
saying, hey, no one's written Fit for the JavaScript
framework, so I'm going to do that.
But then I realized the [UNINTELLIGIBLE PHRASE]
because I needed clicks and verifies and [UNINTELLIGIBLE]
and different things like that.
So it was kind of inspired by Fit, but then it
became its own thing.
And as you can see, it can handle-- well, I thought it
could handle pop-ups.
I guess it can't handle closing them in this demo.
But all the things that AJAX is, not too many
tools can do this.
And so this was actually running in Safari, which I
don't think any vendor can do.
There might be some open source tools.
But anyway, that's just it by itself.
That is the original version.
Now let me show you actually the Firefox IDE.

First I'll show you the little Django app.
Let me close this.
[? I'll click ?] it [? and want ?]
[? it to run again. ?]
OK.
This is your administrative interface.
Maybe I don't want to record it just yet.
Keeping it simple.

It says my IP connections are down.
Rats.
All right.
OK, so I was going to show you the Django application.
But it looks like it's not working, so I'm just
going to move on.
I will show you the Selenium IDE, and that perhaps at least
can record a couple things on the login screen, I swear it
was all working 10 seconds ago.

So it's recording everything.
So it records the URL, the base URL.
And in fact, I type nothing into those user fields.
I can type admin, tab out.

Click Login.
This is going to hang.
Interesting.
So this is the, again--
I guess the technical term for this kind of style of test
would be keyword-driven test. Maybe if you squint your eyes,
you can say it's a domain-specific language.
I don't know.
But the fatal flaw, I think, to this kind of format is that
the source code is HTML, and that's ugly.
And I hate Ant, so I have a little bit of a dueling,
double-think logic there, that I hate Ant, yet this is OK.
So it's fine, because in this format you actually can write.
I think--
stealing the ideas of Ward.
I guess I didn't really think much more like,
[UNINTELLIGIBLE]
Ward liked it, so sure.
He made the argument you could write these tests in Word
tables, or Excel, and this is nice for business analysts and
things like that.
So I said, OK, yes.
Sure.
But also because it's so stupid simple, it's easy to
compile down from Java to this.
So a lot of-- actually people in the very first months of
Selenium, they didn't like the tables.
They used JSPs or PHP to compile down the HTML tables.
And it was pretty simple.
But since then, you actually can record--
you can record, actually, in any format, but we have this
interesting thing in the Selenium IDE.
You actually can change the format to the supported
languages we have for RC.
So not too many people know this.
I think they see Selenium HTML tables and all of that stuff.
And then RC is really hard to set up.
And how do you go from table to Java?
Well, you actually can record your test here,
and then just translate.
It's got some regex stuff behind the scenes to Perl and,
of course, my favorite.
So yes, that's interesting.
So then you can just cut and paste this into a proper IDE
and start massaging this down into a real test.
Another reason why people didn't like so much the HTML
tests, if I can--
I wonder if I can get back here.
We're not Turing-complete.
It's an imperative language.
You need to do one step at a time.
Kind of regressing, I guess, as state of the art languages.
The true proof that this was kind of wacky was when someone
submitted the goto command to the project, saying oh, we
need loops.
Here's a goto function.
You're like, goto previously.
No, no, no.
So we very deliberately have kept Selenese, I guess it's
called, a stupid language.
Because if we added ifs and loops, then, well, behind the
scenes it would start to look like Ant.
But then also because it's a hierarchical structure, I'd
also be fulfilling Greenspun's--
I'm implementing Lisp all over again.
AUDIENCE: [INAUDIBLE]
JASON HUGGINS: What was that?
AUDIENCE: [INAUDIBLE]

JASON HUGGINS: Yes.
Yes.
Yes.
So one of the other things I'm actually playing with, but
it's not fully baked, is some step between full-scale Java,
Ruby, whatever and [? these HTMLs ?] is actually
being able to just write your tests in just straight
JavaScript.
That's not so easy, because JavaScript, as Joe was
pointing out yesterday, it's single-threaded, so you can't
run JavaScript and then stop, like when you say,
go open this page.
You can do that through AJAX, but you can't do it in the
actual window.
Well, if you're interested in writing your tests in
JavaScript, come find me later.
But it's an interesting engineering task.
So that's some of the things.
I'll just briefly show you--

I was going to show you some of the syntax, I guess, for
the Remote Control, so you actually can write it in the
full unit test framework, using the same kind of API.
In Python, Java, C#, Ruby--
the Ruby example isn't as [? advanced, ?]
like a Java server.
Spaces, I think it's going to the page--
[? test ?]
AJAX, and then Perl, so those things.
So I kind of promised Paul Hammant that I would at least
cover Remote Control at some point in the talk today.
So there you go, Paul.
All right.
So now onto this crazy stuff that I wanted to do.
So here's the flow.
We actually called Chris Briesemeister and Andy
Dombrowski in Chicago, working on a project, WEA.
And they have Selenium well integrated into their whole,
continuous integration environment.
And they call it the two tier flow.
Sorry I don't have a slide, but I'll just gesticulate.
They've got a CruiseControl build automation number one,
up here on the top of the stack.
Tier one.
And then tier two is your functional test stuff, your
Selenium stuff.
So the basic flow is developer checks into CruiseControl one.
That runs unit tests.
And then that deploys a test server, which then triggers a
second tier build.
And it's important to do a second tier build because
developers only care about unit tests.
So you want to give them the feedback right away.
And also, function tests are slow, so that's a good
place to cut it.
That second tier--
actually, I think on their project it launches the
browsers and does those things.
My architecture is a little bit different.
And what I want to do today, going with the mind of the
browsershots.org--
I kind of like their idea, where you've got these
servers, but somewhere on the other side of the planet,
you've got desktops that will phone home, and see if there's
a request, and do it for you.

Yes, so I have that kind of a two tier thing, but actually,
that second tier is really listeners on the desktops
waiting for a new successful build from Buildix.
Let's actually not go here.

Let me show the Buildix home page.

Yay.
All right.
At least that works.
All right.
So this is your Buildix starting page.
Oh, let me maximize this.

It's pretty nifty.
You can do some user management goodies, I hope.
Oh, I hope I wasn't looking at a cached page.
I may have. Darn it.

That stinks.

Don't know why.

OK.
Yes, this is really crummy.

So I'll have to do more charades here.

What was that?
AUDIENCE: Can you plug in to the left?
JASON HUGGINS: Did I plug in?
AUDIENCE: Did you plug in to the [INAUDIBLE]
JASON HUGGINS: No, I've actually got my own little
virtual network right here.
What I've actually seen happen before is if the cable gets on
a power cord, it heats up and then it loses the connections.
Or it unplugs.

The first thing to ask.

OK.
Bear with me for a second.
Yeah, total buzzkill there, sorry.

Yay.
All right.
We're back on.
OK.
CruiseControl.
This is what I want to show you.
Yes.
All right.
Status page.
They also have a nifty little CruiseControl widget.
The subtext for this talk, it's cool to buy a Mac.
Anyway, so I won't play around with the dashboard too much.
But this is good enough for now.
So Buildix comes with a couple projects, math, config, and
stuff like that.
So I've got two builds here, basically.
TEA, Time and Expense Application, that was the name
of [? the little ?] project, and then [? TEA screens ?] for
recording the screencasts.
I have my build, actually, for the purpose of the demo, so
long as my cables are plugged in, to check every 10 seconds.
You might not want to do that on a real project.
But anyway.
So just keep this in mind.
I'm actually going to trigger a build.
I'm not going to go and do a test driven or actually edit
code, in the interest of time.
But just keep this in mind-- tea-19.
If I click on the home page, you can actually go back and
look at old builds.
The previous build's kind of hard to see.
But all I did was hit buildtrigger.txt
to trigger a build.
So I'm going to go trigger another build.
And hopefully, tea-19 will go to tea-20.
[INAUDIBLE]

I'm just going to change some white space.

The developer tools--
it's kind of interesting.
I really miss TortoiseSVN.
Just being able to be plugged right into Windows Explorer.
They don't have that.
So this is serviceable.

So I'm going to just commit my buildtrigger.txt, triggering
another build.

All right.
Now let's go check CruiseControl fast. Aha.
It's already caught it.

And if I hit-- ah, it's building.

And it's done.
And it failed, probably because of the
IP addresses again.
So let's go take a look at what happened.
Actually, I can just click in down there.
It failed.
No route to host, so good thing I have some pre-canned
stuff in there.
So let me actually show you what that build was.
It was pretty simple, because I wanted to keep it simple.

Go to the source repository for the project.

CruiseControl build, very simple stuff.
I actually try to use Ant as least as possible.
So I SSH into my Django host and do an Apache stop, if you
guys can see that.
Then I do a--
[? really ugly ?]
[? stuff-- ?]
SSH in there again, and do a Version Up update.
Then I kickoff that Python and database refresh script, and
then I restart Apache.
So since my IP addresses are messed up, the build failed.
But what should have happened is meanwhile, on the virtual
machine ranch, I would have had this listening guy--
I guess I can scroll up here--
he was waiting for quite a while, at some point in the
past when I was testing.
And maybe I'll virtually show you what was going to happen.

Start [? demos ?] here.
All right.

It was waiting, waiting, waiting.
And then it found--
I used the RSS feed of CruiseControl, all the build
status, and looked specifically for a past or a
passing build.
That actually then triggers off the screencast stuff.

So then that calls pyvnc2swf.
And that was interesting, because pyvnc2swf is a screen
capture recording tool, but I found out in the course of
trying to do this demo, that it doesn't have a timeout.
So I actually had to hack in some launch this process, then
launch another process listening on that process to
kill that process in 30 seconds.
That wasn't fun.
And I actually couldn't even get it working in Windows in
time for this demo today.
That's why I just keep zooming past this one, because I
didn't get the screencast in But I did get
it working in Ubuntu.
And I did get it working in Safari.
Yes.
So this counts down and it writes the file, and it
actually commits the file to the repository.
And then it waits and waits and waits again.
So actually, let me show you some of those movies
that are out there.

Go away.

TEA screens.

We get to look at the time line.
So after it builds, then eventually these screencasts
get chucked back in.
So I had a user named Mac.
He had access to update the Safari screencasts.
I wonder if any of these are good.

It actually looks pretty ugly, going straight in from here.

Original format.

I forgot to turn that off.

Anyway, so it actually records the movie.
And this is me trying to get ready.
I'm actually just running through tests.
Maybe I should look through that Ubuntu one as well.

Yes.
This wasn't that interesting.
Ideally I would have had a test of that Django
application, wrote it through the IDE, checked that in, and
then see these virtual machines record that same test
in those various browsers on those platforms. And check
those in, and then all you would have to do is when you
commit it, then go straight back into Trac.
And then just wait for the movie to show up.
So this is all these things, just to kind of give you kind
of a backdrop.
Of course, Selenium is pretty stable.
It's been around for two years.
I think, actually, our two year anniversary was like last
week, I think.
All this stuff I'm showing you is kind of like mad scientist
research and development kind of things,
trying to get going.
So this is kind of flaky, but it's an extension of what is
possible when you use things like Selenium.
And when you embrace something like this, when you start with
a screencast and work backwards from that, I guess
you have to kind of agree that a screencast might be useful.
Maybe this particular one is not useful.
But your user documentation, so David Hansson, every time
he comes out with a new, nifty version of Rails, he doesn't
have to go and do those things.
Also, taking it one step further, Macs, especially, you
have another reason to buy a Mac, because it's cool, has
really good speech synthesis.
So if you can imagine, again, another thing I would have
loved to have done in time today, but something that's
actually made possible by the fact of Selenium Remote
Control when it's on the server side, you can go do a
server-side process.
And then you can do something in the browser, or do
something on the client, do different things.
So as an example of that, perhaps you could actually tie
in the voice synthesis from Apple, and make an actual more
interesting screencast. So imagine-- close your eyes.

[? [INAUDIBLE] ?]

So imagine going to the front page.
[? [INAUDIBLE] ?]
Hear her?
[? [INAUDIBLE] ?]
Can you hear that?

[? [INAUDIBLE] ?]

Well, that was my--
well, the sounds over, it's part of the demo.
I could do it again.
But use your imagination in the sense that you could use
voice synthesis as the next--

Coming in, I saw the Steve Jobs demo of Leopard coming
up, next year.
The voice synthesis is even better.
Actually, you can't tell it's a robot.
So imagine recording this to MP3 concurrent to the running
of the tests.
And if you use Selenium Remote Control, you can time it
specifically.
Queue the sound, wait for that to be done, then do a couple
of interesting commands to the screen.
And also, if you go again with the focus of doing it through
screencasts, you keep your tests--
yes, they're brittle, but then they're also your
documentation and you want your
documentation to be brittle.
But also because people potentially will use this,
potentially if you do it right, this could be a good
example as a marketing tool.
You will make write short, pithy tests.
And so kind of going down this track, you might actually make
the argument that you'll probably use Selenium the
least, but it's flashy stuff.
Like the actual testing of your business logic, you would
be doing that in the protocol layer or somewhere else.
Unit tests, of course, millions of them.
And this would be kind of the flashy stuff
that you could automate.
So those are things.
Yes, without the IP, I could go hunt down there, but I
really kind of killed the buzz here, so perhaps a little bit
early, but I can open it to questions and discussion.
Antony Marcano, testingReflections.com.

AUDIENCE: Hi, Jason.
Thanks for the announcement.
Well, as you know, I've been using Selenium for a while,
and I'm very, very inspired by what you've done today.
I have a feeling that maybe in a couple years' time, this
will be some sort of multi-million [? power ?]
business, that we can all submit our requirements and
you'll go off and generate these videos for us.
JASON HUGGINS: Unfortunately, no.
I can't.
We haven't figured out how to make money on it, because it's
open source.
And I just told you how to do it.
So no.
AUDIENCE: I'm sure there will be one.
Talk to me afterwards.
Anyway, what I'm interested to find out is obviously these
videos can get quite big.
And if you're doing this as part of every build--
JASON HUGGINS: And checking it into Trac especially is--
or not Trac, Subversion, is a terrible idea.
It was the simplest thing that could work.
Is that your question?
AUDIENCE: Well, exactly what I was getting to, for the time
being, obviously, they're quite big.
Can you see a way of this actually being perhaps used
just on an overnight basis?
And so I could come in the morning, for example, and I
can just say, the build failed, and it failed because
one of our Selenium tests failed.
OK, I'm going to go playback that video now.
And do you think that's going to be manageable for a lot of
applications in terms of integration with Subversion
and/or disk space and storage and so on?
JASON HUGGINS: Honestly, I have no idea.
But I went with the simplest thing that could work.
I knew that protocol, it's WebDAV. And it was check out,
check in, all of the authentication
was just all there.
But yes, because Subversion stores every copy of it, you
might only want the last one.
And also, there's something else that someone brought up.
You only want really to see these if they failed.
Who's going to just watch it run again that it passed?
Well, if you go with the thought that your potential
users, as a marketing tool, those are useful.
But yes, something better would have been
to maybe FTP it.
FTP it over the overlay, something like that.
Those are things I'm still trying to figure out.
I really like the ADAM API.
Again, thinking of kind of distributed grid, of having
computers all over the world kind of checking in, doing the
recordings.
And that's a good standard, that I'm really kind of
eagerly following.
But as far as actually what's going to be best on the server
side of where things get stored?
I don't know.
Yes, buy a big hard drive.
AUDIENCE: I think what you've done is really cool.
It's innovative and I think it's got great potential.
So don't give up.
And I hope we see the complete version of the talk in another
conference very soon.
AUDIENCE: Hi.
This is Ryan Campbell, with Red Hat, over here.
So one need I think a lot of people have is for ensuring
visual consistency between different browsers, so that it
generally looks the same across browsers.
So one way to kind of extend what you're doing and
incorporate the small multiples that were
[? tough ?] to select--
take these screencasts and put them in a grid, and play them
together, all platforms playing at once, to allow to
you to see how the platforms differ visually, and let you
zoom in on ones that are aberrant or
something like that.
So it's kind of a cool idea to build on top of that, to see
the cross-platform visuals.
JASON HUGGINS: Yes.
Actually [UNINTELLIGIBLE]
I hadn't thought about that one.
Put it up on the screen, during a party.
Psychedelic.
Yes.
No, it's a good idea.
Thanks.
Any other questions?
We're in the final stretch here.
I'm amazed I don't see too many people sleeping.
Yes.
AUDIENCE: What's the--
I know you're from a small company with head office in
Chicago, you may have heard of.
What's the most complicated application of this that
people have done in production?
JASON HUGGINS: What's the most complicated?
I don't know.
I haven't seen them.
I know we have lots of--
well, to name drop without namedropping, many Fortune 500
are using Selenium in their products.
I don't know if I can actually mention any of them, but yes,
we've got some very interesting--
yes, talk to me later.
But I know there's big companies.
So if you go with the logic that big companies have big
applications, and they're actually coming after us to
see if we could provide support contracts to them for
that, then I guess ergo, it's being used in a complex
application.
But I'd have to go track those down for you.

Probably the least used is the Django interface for testing.
AUDIENCE: How maintainable is this in the long run?
JASON HUGGINS: How maintainable?
AUDIENCE: Yes, [UNINTELLIGIBLE] changes and
[? rebrand. ?]
JASON HUGGINS: How maintainable
are Selenium tests?
I can answer that in a couple ways.
The HTML tests themselves, like Fit tests, or Fit style,
they're unmaintainable, because you can't--
I'm not sure if I made that point.
Or I guess I did.
You can't do ifs, loops, refactoring into functions,
and common stuff.
So those are unmaintainable.
But they're good enough in small doses.
If you use Selenium Remote Control, and then you write it
in Java, then it's as maintainable as
anything else in Java.
Or as not maintainable, as someone might say.
AUDIENCE: So what happens when the GUI changes?
JASON HUGGINS: The GUI changes.
Well, right.
Tests are brittle.
I'm making kind of the contrarian point though, that
when you deliver documentation that's
like the bane of everyone.
So the app developers, testers, and don't forget the
guys who document stuff, because I think they're kind
of third, like the forgotten step-child of software
development, the people who have to
document all this stuff.
There's always that problem between the docs being out of
date from the code.
In that sense, you want to couple the
documentation to the code.
You want it to be brittle.
When code changes, that changes.
Otherwise you don't know.
That's a big complaint about a lot of open source projects.
Oh, yes.
I found the documentation.
But it's 10 years old.
So you kind of go with this trade-off.
It's going to be brittle, but then it may be, in that sense,
you use it less for the things that need to be brittle, the
documentation.
But for stuff like you're testing your
business logic, yes.
I learned the hard way.
I used, when we first made Selenium, Selenium was the all
being function testing tool.
Everything that was not the unit
test library was Selenium.
And I think I've realized that over time I'm using it less
and less and less, but for more interesting things.
So I don't know if that answers the question.
AUDIENCE: Yes.
JASON HUGGINS: OK.
Yes, use it less and then it makes it easier.

MALE SPEAKER: All right.
Jason, thanks very much.