GTAC 2010: Flexible Design? Testable Design? You Don't Have To Choose!


Uploaded by GoogleTechTalks on 07.12.2010

Transcript:
>>
RUFER: Okay. So, just to quickly define some terms, when we're talking about flexible design
here we're talking about code that's easy to modify and extend. Pieces of behavior in
the code, you want to be able to reuse elsewhere and in order to extend your system easily,
you need to be able to understand it. When you make changes in the system you don't want
those changes to be intrusive changes, and as you expand the system you don't want to
explode complexity. Any number of reasons why a flexible design matters but a few that
we usually like to focus on are when you're expanding teams you need to be able to onboard
developers quickly so they have to come up to speed on the code. When market conditions
change, you're going to have to respond quickly with unanticipated feature changes. When you
introduce new functionality, you need to be able to do so safely without introducing a
bunch of bugs. >> BIALIK: So when we're talking about testable
design during this talk we'll be talking about unit testing. Some of the ideas do extend
to components and subsystems but we'll focus on unit testable code. So what does it mean
to be unit testable? We need to be able to instantiate a class and its collaborators
in an easy manner. We want to be able to set it up into the desired state and then drive
the code through each of the execution paths and then witness the effects to ensure that
the code is behaving the way that we expect it to. So, what makes a unit test good? Well,
we want them to run quickly and in parallel whenever we'd like them to do that. We'd like
them to be hermetic, meaning they need to be isolated one from other. When we have an
error, we'd like to be able to find exactly what causes that error. We don't just want
to know there's a regression we want to be able to pinpoint the code that's causing the
problem. Good unit tests are easy to write and to read, and they validate not just the
happy paths but also corner cases, exceptional conditions and error cases. So I think we're
here among our fellow test engineers and people who are interested in testability, but the
reason that it's important is we want to isolate pieces of behavior for validation. When a
test breaks, we want to see exactly what went wrong and we don't want to have to jump into
the debugger to try to solve the problem of, "Why did this test break?" At Google, we have
a motto, which is, "Debugging sucks and Testing rocks!" So, brief aside. We're not trying
to say that end-to-end testing isn't important, it absolutely is. We love unit tests but we
also understand the need for end-to-end tests. >> RUFER: So, as we proceed here, what we're
going to do is look at a series of common design choices that need to be made. For each
one, we'll introduce a situation. We'll take a look at one possible design alternative
that has poor flexibility and poor testability. We'll explore why and then we'll introduce
a second design alternative that improves both of these characteristics.
>> BIALIK: So, we'll start off with a running example. This is a hypothetical application,
the Google Cacophony. It is a way for people to collaborate on musical compositions and
it just do a lot of things; we need to process music change requests, parse user preference
files, apply some audio filters, turn this into midi files that are playable and send
results the users. >> RUFER: So, the first common situation,
we have to answer the question, how large should our classes be? We often deal with
developers who ask--who say, "I really don't want to introduce a lot of little tiny classes
into my system unless there's value for doing that."
>> BIALIK: So, our first design is the not recommended design, and that is put several
behaviors in a huge class. We've got a MusicServer and you can see by this comment, it does--thank
you. You could see from this comment it does all kinds of things. It's processing music
change requests, parsing preference files; everything that we have under the sun is in
this piece of code. >> RUFER: So, this isn't a really good choice
in terms of flexibility. Reuse is inhibited because any piece of behavior in the system
is tucked away inside this one big class. If you'd like to use it anywhere else, it's
unavailable. Also, suppose we're wildly successful and we branch on beyond midi files, we want
to start adding MP3s, waves, our parsing is going to get more complex, we've got audio
filters that we're adding everyday, new instruments. That complexity has to go somewhere and if
we've got multiple responsibilities in the same class, that's where they're all going.
Also, conditional complexity is likely to explode inside this one large class. Let me,
before I move on to some other flexibility concerns for this design, let me just take
a brief aside to say, global data is bad. Do we all agree we'd like to minimize the
use of mutable global data in our system? I see some hands up. Okay, yes, reduce the
use of global mutable data. But what's the problem with global mutable data? The problem
is what Michael Feathers calls a "spooky action at a distance." If any code throughout our
system has access to be modifying this data then it means the code that depends on the
data is difficult to understand because the data is going to be shifting underneath it
and it's difficult to tell who did that shifting, where did the change come from. Why am I bringing
it up? Because if we've got multiple responsibilities housed in a single class then we've really
created a system where the fields in that class are essentially global. Or maybe it's
just a portion of our system, but for every responsibility that has access to every piece
of data in this class, any method could change any field. So, we're in kind of the same situation
as with global data, we were in the large. We've lost application within this space that
we've created. It's the same situation for private methods. If any method in this large
class can call any private method we have to ask ourselves, "How private is private?"
>> BIALIK: We also have to face that the testability of the system isn't very good either. Just
imagine your input is a change encoded in an RPC and your output is and MP3 file. So
trying to unit test this is really, essentially an end-to-end test. And we have this input
coming in, various things are happening inside this large class, the output is an MP3 file.
How do we validate that? We're probably going to make some golden files, some well known
files that are good. We're going to have to compare these two files to say, "Is this the
right answer?" And any time we have a regression it's really tough to tell, "Where do we go?"
"We go right into the debugger." "Where did the problem happen? Well, let's whip up the
debugger and we'll find out where it is." Also, if you make intentional changes that
should alter the final MP3 files, you have to create new versions of your golden files.
And if you have to change a lot of those, that's going to be a real pain. Another thing
is the notion of safety. When we are changing our code, we want to address change the code
and we'd like the test to be a safety net. If we have to change both the code, the production
code, and the test code together, then we've lost that safety net aspect. We may make mistakes
in doing these changes in two places. >> RUFER: Okay, so, let's look at second design
alternative instead of one large class. Let's define crisp small classes with distinct behaviors;
RequestHandler, PreferenceReader, PreferenceParser, MusicMixerp; each doing just one thing. Sometimes
this is called singular responsibility principle, sometimes it's called separation of concerns.
We've improved flexibility with this design. Each of the classes is easy to step back and
understand as one conceptual unit. In the larger class where we had all these behaviors
mixed, we lost the power of the object oriented programming. When we split the behaviors up
into crisp distinct units, we get that power back. Object-oriented languages generally
provide us with natural features for encapsulation, and polymorphism. It lets us--when we have
variable behavior to accommodate, which is always the case with flexibility. It allows
us to further subdivide our behavior into siblings and in inheritance hierarchy that
we can install or even swap dynamically. >> BIALIK: Our testability is also improved
now with our smaller classes. The test can be more focused on each individual class.
It's much easier to test unusual or exceptional behavior. We can drive our small classes through
the various paths, and we don't have to go that kind of big end-to-end. The input for
each unit test is really easy to supply. We don't have to start with the change request
and then our outputs are an MP3 file that we have to check against the golden file we'll
have smaller targeted focused tests. An example would be, "How would we test the audio filter
selection is always correct?" With the first design with a large class with a lot of behavior
we would need to construct golden files for every different kind of audio filter selection.
It's going to be difficult to do and it's much easier in our second case where we had
these smaller classes, you can just test one piece of it. We can test that a filter selector
is returning an appropriate filter depending on the filter specification.
>> RUFER: Okay, let's move on to a second common situation. Classes need configuration
data. That's true of just about every system, right? The question though is, "How should
we supply this information to each of the classes that need it?"
>> BIALIK: So, in this case, we're going to have a user that directly reads data from
a file. So internally, the user is reading a file, parsing it and creating either a configuration
object or some kind of data structure from that information and a user has other responsibilities.
This isn't the core responsibility of it so this is a smaller violation of single responsibility
principle. But we see this so often that we wanted to call it out specifically, and you've
probably all seen this kind of code. We've got a user, it has preferences field. We take
the preference file, we parse it and then we store the resulting data into the preferences
object. Later on, we do something with the preferences.
>> RUFER: Okay, so this gives us some flexibility concerns. If you want to change this file
format, how do you--how do you change it? You have to go into the user code and make
intrusive changes there. If you want to move on and supply--support multiple versions of
preference files, you're going to be adding conditional logic into your user. User is
tied to the file system. Suppose that our Cacophony service becomes wildly popular,
many users and we need to offload preferences to a preference server, we want to communicate
with the preference server via RPCs. Again, we're going to be going into user, we're going
be making intrusive changes so that it can depend upon the RPC mechanism for its preferences
instead of the file system. And suppose that elsewhere in the system, these preferences
would be useful. Well, we've buried the reading, parsing, validation inside the user code and
inhibited reuse with this design. >> BIALIK: This design also has poor testability.
We wanted our tests to run fast but we know that tests that hit the file system are slow.
And to setup our test, we're going to have to have properly formatted files, one for
every kind of preference or every combination that we'd like to have. And then if the file
format changes, let's say we'd have to have a new piece of required data, we have to go
to every test file and modify it. >> RUFER: Okay. So, preferred design for this
instead of having user tied directly to files, we have dedicated code elsewhere that reads,
parses and validates the file and we install the constructive preferences into the user;
direct injection. Now, so far all we've done is we've moved the problem from user somewhere
else and later in the talk we'll be explaining that somewhere else and how to solve that
as a problem. We've improved our flexibility though. If we have changes to the file format,
user is insulated from that because it only sees the constructive preferences. If preferences
come in from another source, user is insulated from that, it remains stable.
>> BIALIK: And now with our new design where we don't have to go to the file system, our
tests are going to run a lot faster. It's much easier to construct a preferences object
and create different variations for testing in memory. And also, our tester no longer
tied to that file format. So if the file format changes, our tests don't actually have to
change. As an aside, this is true for general kinds of input. The same reasoning applies
for files as for raw sockets and RPC mechanisms, et cetera.
>> RUFER: Okay, let's move on to yet another extremely common situation. We have stateless
services and utilities throughout our server and classes throughout the server that need
access to those. The question is, "How does a class that needs the service get access
to that behavior?" >> BIALIK: One choice that we often see being
made is to access the service via static method. It often seems like a good idea at first;
"Hey, that's really easy." The service is globally available. We just can call the static
method, we don't have to create an instance and that's going to be really easy to test,
right? In this example, we have a musical scorer, and when the score is changed, we'd
like to inform authors via email. So we came up with this great idea to have an email sender
and it'll have a static send method. We can just easily call that and give it the message.
>> RUFER: Okay, we've got a flexibility concern. What happens as we want to support additional
ways of notifying an author? We were notifying by email before but, now, some users want
text messages; others want a Google buzz or a Twitter tweet. Here is a typical code that
we see feature developers write to solve this problem. We're switching on the preference
of the author--of the musical author whether they want text message sent to them, whether
they want email sent to them, and for each of these different possibilities, we're hard
coding connection to different static service. So, this isn't really a good choice because
we've increased the complexity and there's really no easy way to use Polymorphism to
solve the problem. What we'd like to do is we'd like to have just a single interchangeable
way of doing send notifications. >> BIALIK: This design also has poor testability
implications. So naīve testers might think this improves testability. "You don't have
to instantiate the service, you can just test it." But the problem is the consumer of the
service can now not be tested in isolation, it's directly tied to that service. And often
these services are complex or expensive, and we don't want to do that in unit tests we
said we want our tests to run fast. Also, trying to prevent an email from actually being
sent in this first example, takes extra work. Typically, we'd have to put some code inside
the email sender that's for testing, have a little flag that says, "If testing, don't
actually send an email," that's pretty bad. And we'd like to--a brief aside to have a
caveat here, we're not talking about simple stateless utilities. There are things that
are fine to have, like math absolute, is totally fine as a static utility we're talking about
more heavy weight services. >> RUFER: Okay, so instead of having these
brittle static connections, our preferred design is to make the service instantiable
or at least to have a class that we can instantiate that access the service on behalf of the consumer.
This allows us to take advantage of an inheritance hierarchy to unify the variant behaviors and
we can inject a subclass of a sender interface into the musical scorer. The code looks like
this. Somewhere else dedicated to understanding an author's preference we collect--we construct
a sender subclass and then musical scorer is very simple. It just takes a sender; it
doesn't care which kind of sender is installed. Sendupdate no longer has the conditional logic
that we saw before, it just--whatever sender was installed in its constructor, when it's
time to send an update it sends the message through that sender. And again, at this point,
it just looks like we're moving the problem one level up but we'll be talking about that
in a minute. So flexibility has improved now. Musical scorer doesn't know every different
way in the system that an author might want a notification sent to them. As I said conditional
logic has been removed and adding new sender types is not going to make us go and make
intrusive changes in the musical score, it remains stable to that kind of change.
>> BIALIK: Our testability has also improved now. Now, we have a scene where we can insert
fakes or mocks where we'd like to. This helps us avoid those expensive operations that we
had been statically tied to things that may take up a lot of time or memory, or have avoid
the undesirable side effects like actually sending an email every time we run our test.
We also can simulate exceptional cases and error cases when we have fakes or mocks instead
of just, depending on the service to maybe fail for us sometime, we can inject the fake
that simulates a failure. >> RUFER: Okay, onto another a very common
situation, getting data to consumers. So we have classes throughout the system that needs
some kind of data. Often it comes from command lines, initialization files maybe directly
from a request or anything derivable from those sources maybe via data store. And each
of the classes in the system really only needs to work with one or a few maybe zero kinds
of this data. >> BIALIK: So what happens when we very first
start with the system? We might just weave one data item from main down to the classes
that need it. But then our system gets larger, and now we've got second and third and fourth
parameters and we need to weave those all the way down into our various classes. As
the system starts using more data, we start tacking on additional parameters, and it gets
pretty ugly pretty fast. We don't want have to pass, you know, 50 parameters in every
method call. So, there are few designs that people tend to use for this and we don't recommend
any of these, as the systems grow, we see--sometimes people will bundle their data into a context
object that's passed throughout the system, or chunks of data are packaged together into
classes that are already widely known in the system. And then the other is the bucket of
static data, which is a problem that we've already discussed in terms of global data.
>> RUFER: Okay, so, Design 1A is the context object. It's a big collection of data that
we end up passing throughout the whole system. Often we have in a--if we think of our system
as a large aggregate structure, this context is passed from parent to child to grandchild
down through the system. Many of the nodes touching it only need to touch it to pass
it around. Others receive it, this context object, and they reach in to the content to
grab the data that they need, there's some flexibility concerns with this design. So
the context object is so widely known throughout the system that it might as well be global
data. Although problems that we said, we had with global data are also problems with this
extremely widely known context object. We'd also like to build our system in such a way
that we can look at a class, look at its constructor and public methods and be able to understand
who--what other parts of the system it collaborates with and what kinds of data that class makes
use of. When we're passing around this large context object, we've lost some of that. You
don't know whether a class receiving this context object, uses it at all, just takes
it so that it can pass it down to child-class. And if it does use it you don't know why just
by looking at the surface of the class. You don't know whether it needs access to a country
code or not. We've also inhibited reuse severely because if we've got classes that depend on
this particular context object that maybe would be useful elsewhere or in other systems,
it's really difficult to reuse them there because those other systems aren't going to
use the same context object that the class that we're passing this through is brutally
tied to. >> BIALIK: So the other choice is what we're
calling holders and basically it's overloading a well-known class to carry extra bags of
data. We often see this was something like a user, where this user object is needed in
a lot of the system and we pass this data clumped up with the user. And what happens
is we add a little bit of data to the user and then that starts to get pretty big, so
then we split that off and we say, "Okay, well, now our user has some preferences,"
and that starts to get pretty big. And so we say, "Well, let's have a location data,"
and then inside of that, "Well, you know that's pretty big, let's have it have the country
code." So, then when we have this user being passed around, classes like account that require
a country code have to dig it out of the user. And what this looks in practice is an account
takes a user, it wants a country code, but it goes to the user and it gets the preferences
out, and then it takes the preferences and it gets the location data out and then it
takes that location data and it finally gets the country code and that's the piece that
it really wanted. >> RUFER: Has anyone seen this code? Okay,
everyone in the room has seen this code. Okay, this code has flexibility concerns. I already
said public methods don't really reveal what's going on. Also, we've got this code that Tracy
just discussed that digs down into the structure that we're passing around. It's often duplicated.
If we've got two places in the system that need the country code, we often see the same
dotted, dotted, dotted code to dig it out, and this digging code also locks the structure
in place. So, if we were to decide that our system would be better factored if country
code were stored in another way or somewhere else, we've got maybe 50 places in the system
that are hard-coded to this dotted representation of the structure that it's currently in, refactoring
is going to be severely inhibited. >> BIALIK: This is also a problem for testability
in both of the cases with the context object and these holders. Our tests are unnecessarily
complicated. We have to do a whole lot of setup to get these pieces of data into this
nested location only for the purposes of allowing the class under test to basically dig it all
out. What that looks in practice is we've got a test where we've got an account and
it needs the country code but it also needs to get out of the user. So we have to make
a country code, a location data, install the country code and the location data, then we
have to make preferences because the location data is part of the preferences, then we have
to make the user and install the preferences in the user, finally, we can give the user
to the account. And all of this just so it can do that dotted digging out of this data
to get the country code. >> RUFER: And we tend to see this kind of
code in tests repeated over and over again. So, there's a preferred design. Instead of
nesting the country code way down in this structure,, we're going to say, "We should
just pass the country code directly to the class that needs it," in this case, the account.
And that may sound like I'm just being flippant, but at several points we've said we're going
to get to something later and the time for later is now.
>> BIALIK: So, we're going talk about system assembly and we're going to use a different
example than our Google Cacophony server because this one is something that people understand,
how things are put together. So, if we're going to try to construct a car and it needs
a windshield and an engine, and an engine needs spark plugs, we're going to look at
two ways to assemble the system; one will be top-down and one will be bottom-up. So,
the earlier design choices were poor and the reason that we ended up with these is due
to this following structure. And it's the top-down assembly where we say, "Okay, we're
going to make a car and inside the car's constructor, it's going to construct its own collaborators.
It's going to new up an engine, store it in a field; it's going to new up a windshield
and store it on a field." What happens when we say new engine? Well, engines need sparkplugs.
So in the engine constructer, it's going to conveniently make its own sparkplugs, put
them into a set of sparkplugs. Now constructing a car is really easy, right? All I have to
do is say, "New car," and I get a fully assembled car and get the full object graph. But the
problem is that changing the parts inside the car are really hard.
>> RUFER: Okay, so this a brutal way to assemble our system. Suppose that we decided that we
want a high performance sparkplug instead of a standard sparkplug. Where do we have
to make that change? We have to go directly into the engine. The engine needs to know
there are two kinds of sparkplugs, it's going to have to conditional logic. In this case,
in its own constructor to make a sparkplug selection, and in some cases, it has to do
even more work to additionally configure its own collaborators. Just moving sparkplugs
one level up to car doesn't help because then car needs to know about both kinds of sparkplugs
to be able to pass those down into an engine. >> BIALIK: As you can imagine this also has
an implication for testability. When we want to use a car in our tests we get the entire
object graph, we don't have any choice, that's what you get. And that might be expensive
to make and so every test is going to have to create this entire object graph even though
we maybe don't really want that. If we want to have test versions of our collaborators,
there's no scene, there's no way to get those in there. Everything is taken care of under
the covers, we're stuck with the standard windshield, the engine and the sparkplugs.
>> RUFER: Okay, so, design two, which we prefer, is bottom-up assembly. Instead of having every
class construct its own collaborators we'll have dedicated creation mechanisms off to
the side. In a manual situation, this implies a collection of builders and factories. You
may be fortunate enough to be able to use an automated framework like Google's Juice
or Spring but the idea behind it is instead of creating a parent that creates a child
that creates the grandchild we're going to create the grandchild first, we're going to
start from the leaves. And once we've created sparkplugs, then we can create an engine that
we just passed the sparkplugs into. If we have a dedicated sparkplug factory that knows
how to decide whether we're using a standard or high performance sparkplugs, then it creates
the kind of sparkplugs we need and can install those into the engine; no conditional logic
in the engine, it just uses sparkplugs. Once we've created the engine, we can create the
windshield off to the side and now it's easy to construct a car just passing in the already
created and populated engine and windshield. So flexibility has improved because now we
don't have to make intrusive changes into classes like engine in car when we have variant
behavior to support in their children. >> BIALIK: It's also now easy to test our
pieces in isolation. We're not stuck with that big object graph. We can create pieces
as we like. It's really easy to create fakes and mock collaborators and inject them into
our system under test. So we can make a car with a fake engine that refuses to start and
we can test out that behavior. We can make sparkplugs that actually validate if it was
asked to fire by the engine. This lets us push through all the different conditions,
which is what we want for our tests. >> RUFER: Okay. Let's move on to another extremely
common situation. We have clusters of behavior that change together and multiple sites throughout
the system, perhaps in the same class, perhaps spread across classes that use these clusters
of behavior. An example to make this clear, in the Cacophony server we have different
account types; free, basic, and premium. And depending on what kind of account we're working
with, a MusicMixer may need to be placed some limits. Maybe for a free account, we can only
afford to provide the collaborators with the users out there with a 10-second duration
clip. Basic account, they can get a minute. Maybe it's an unlimited music clip for a premium
account. We may contract with third parties who provide filters and instruments for us.
And depending on whether you're free, basic or premium, it could limit either access to
or maybe the quality of a filter or an instrument that we'll use.
>> BIALIK: So the first design choice is, again, our not recommended choice. We have
a Director class and it has a MusicMixer and an account type as fields. And when we want
to have each of these features we go and we do some conditional logic for deciding what
the account type is and then what the behavior is. So if the account type is free, we'll
ask the mixer to truncate the audio short. If the account type is basic, we'll ask the
mixer to truncate the audio in the longer section. And if it's a premium account, we
won't ask for truncation at all. We also--still inside the Director, we'll have a set up for
instruments free and basic and such, and similarly for applying our echo filters.
>> RUFER: Okay. So this design choice is poor in terms of flexibility. Any time we want
to make a change to add an account type, we're going to be extremely intrusive because for
each clause we're going to have to go and add a new conditional statement for how we
handle mega-premium accounts. The logic for how we handle a given kind of account is scattered
around. So if we wanted to step back and understand how does our system respond to when we're
working with a premium user, you can't just look in one place, you have to look at behavior
that's dotted throughout the conditional clauses. And this kind of code invites bugs. When you're
making similar decision over and over and over again and switching on that decision
you're asking for a mistake to be made. In fact, we just got a call from hypothetical
support on our hypothetical server and they tell us that we've just been charged by our
third party vendor for a hypothetical extra million uses of premium echo filtering. Let
me back up here and say here's what happened in one of these conditional clauses. We said,
"If account type equals premium, we're going to apply the free echo filter and the last
fall-through case, which should be free, is applying premium echoes. So, we made the same
decision in multiple places; in one place, we got it wrong.
>> BIALIK: Testability is also impacted when we want to validate that we're doing the right
thing. The Director has to call each correct Mixer method. And we have to make sure that
we do that for every account type at every feature otherwise we'll have these bugs at--like
we just found. So we need a mock mixer to expect the free or the basic or the premium
versions of every single feature. What that means is a combinatorial explosion of tests.
We've got a cross-product of features and account types. We've got free, basic and premium,
and then we have to cross that with truncation, filtering, et cetera. So a three-way decision
at each of five features means 15 tests. As soon as we add a new account type, we have
another test for every feature. Or if we add a new feature, we need another test for every
account type for that feature. >> RUFER: So we have a preferred design decision
for this and that's isolate the free, the basic and the premium behaviors each into
a separate class and create an inheritance hierarchy to unify the interface to those
classes. In this case, MusicMixer interface with one method per feature. It looks like
this. So off to the side, we're making a selection of which mixer to use. That selection is made
just a single time in one place instead of repeating our conditional logic over and over
for each feature. And then when it's time to use a feature we get to delegate to the
properly selected mixer or cluster of behavior. This makes it really easy to support new account
types because, again, we've eliminated the conditional logic from inside our Director,
all we have to do is create a new kind of MusicMixer and pass it in. We also--with this
design have addressed the consideration of the logic that were scattered about. Now,
we have a single place we can look and say, "How do premium account works?" The Director's
become simple and the bug that we saw a few slides ago is almost never going to occur.
>> BIALIK: This also helps us with testability. We've now eliminated the combinatorial explosion
of tests. We don't need multiple tests to ensure that the Director is calling the appropriate
MusicMixer method because now we've consolidated that into single methods per feature. We may
end up writing a test that validates when we have a premium account we get the appropriate
mixer but it may already be tested by our end-to-end tests. So we'd like to go from
the specific to the general. Why does this work? There are three foundational pillars
of object-oriented code; encapsulation, loose coupling, and high cohesion. And as an aside,
a strong use of polymorphism supports both cohesion and encapsulation. So let's go back
through each of our examples. We had the music server with most of the code in one class.
We said that wasn't very encapsulated at all. As soon as we split that into multiple classes,
now we have better encapsulation and we have high cohesion if we've made good choices for
splitting at that behavior into the appropriate classes. When we had a user reading preferences
file directly, we weren't encapsulating our file parsing or the formattable file. And
so we had a high coupling between the user and the file and its format. When we instead
inject the preferences data right to the user, we've now encapsulated the file parsing and
format somewhere else into a file parser and we've decoupled the user from the file and
its format. We had the musical score where we were using the static sender service and
we ended up with some very tight coupling between the musical score and all of the sender
services. The musical score had to know the concrete type of all of the senders. That's
not what we want. We'd really like things to be loose and so we can easily inject a
different kind of sender, we can create new senders. We want loose coupling between our
musical score and our sender. We want only to know about the sender via the interface.
We talked about context object and holder for bringing data to the various kinds of
classes throughout our system. The problem with the context object and a holder is there
isn't any encapsulation. We're digging into all of these pieces ripping out the parts
that we want. We also end up with tight coupling. We have all of these classes throughout the
system weaving this context object through. They're all dependent on either the context
object, or in the other case, the user. And we end up with low cohesion. The context and
the user are basically dumping grounds for information. Things don't necessarily go together.
It's just convenient to put it there. Instead, when we inject exactly what the class uses,
we don't violate encapsulation anymore. We're not digging into the user to get the preferences,
to get the location, and we also have loose coupling, we only tie classes to the collaborators
and the day that they actually use. We talked about system assembly and the preference for
bottom up. When we have top down assembly with the car creating the engine and knowing
up all the pieces behind the scenes the assembly logic is distributed throughout the system.
And we also have some very tight coupling. A car has to know about all the concrete engine
types because it's going to be creating them similarly for the engine and the sparkplug
types. When we go bottom up and we have factories and builders to help us out, we separate the
way that we build the system from the business logic and we also end up with much looser
coupling. A car only knows the engine by its interface. It doesn't know all the concrete
types of engine. And in the MusicMixer, we had the Director with the MusicMixer and three
accounts. The behavior for a given account type isn't cohesive, it's scattered throughout
the Director and it's also sort of scattered throughout the MusicMixer as we saw there
are different methods for each feature. When we instead go with the three mixer subclasses
we end up with encapsulation. The Director doesn't even know about the account. All it
knows is it knows the mixer by the interface and somewhere else is making the decision
of which mixer to use. We also have much higher cohesion because the behavior for an account
type is together in one of these mixer classes. So the common themes here are encapsulation,
coupling, and cohesion. And if we optimized for these we get an increase in the flexibility
and testability. We end up with units of behavior that we can exchange, we can vary, we can
reuse throughout other parts of the system, and our testing is much easier. We can have
small focused unit tests. We can push our code through the various paths. We can easily
substitute collaborators. We can do fakes and mocks.
>> RUFER: And so working with many people or many teams is that you can't just have
a single canned argument or prepared speech that's going to convince everyone. Every case
is unique. People care about different things. They're in different situations. You have
to tap into the unique situation you're working with in order to actually influence an engineers
design habits. There are many right convincing arguments and not every argument is going
to work in every case with every engineer. You need to understand the developer or the
team what their motivations are. You have to understand something about the context
that they're working in. What has worked well or poorly in the past for them? What kind
of challenges they have in front of them? The first thing you want to understand is
what motivates the individual. Some featured developers, they care about testing. And you're
going to be able to make a strong argument just in terms of how they can improve their
ability to right the tests that they want. Some developers, well, you know, they'll write
a test if they're kind of prodded by you or their colleagues but they don't want to think
about testability when they're designing code. But they are motivated by producing code that's
flexible so you want to understand that and tailor your argument to the individual. You
also want to know something about the particular code that they're actually working on. You
want to take into account how a specific individual learns. Some people are really good with theoretical
explanations and some really, especially a lot of engineers, want to see something very
concrete. They want an example and they'll generalize themselves. If you're working with
theoretical learner, you might need to first establish some common ground like we did introducing
the notion of global mutable data that then allowed us to discuss related cases like multiple
behaviors housed in a single class. If it's an individual that learns best by example,
you're going to look in their code base, hopefully, the code that that individual has written
and find an example where you can point out the kind of design situation that you're talking
about and make your points using their code. Then you want to select the strategy. If it's
someone who can be influenced by testability arguments, look through the code and find
a test that they can agree, they'd like to write but is inhibited with the current design.
Show how the alternate design that you'd like to propose solves that problem. Maybe you
can even work together toward the solution so that they have that sense of discovery
instead of just being told. If it's someone motivated by flexibility, find a feature that
is currently inhibited in terms of flexibility, maybe something that was already implemented
but is really awkward or a feature on the backlog that is going to cause problems that
you can show the difference between two design choices and how the one that you suggest will
allow that flexibility to be implemented easily. >> BIALIK: So in conclusion, we hope that
you'll take these ideas and generalizations to analyze your own code and to apply flexible
and testable design and influence with your teams to do likewise. So we have some time
for Q&A. >> Hi, Russ and Tracy. I have a question with
respect to encapsulation and its impact on testability because encapsulation was one
of the common threads in your discussion through the PPD. So encapsulation itself makes only
a subset of the methods or the properties available as a part of the contractual interface
thereby limiting the testability. So how do we deal with encapsulated classes? How do
we test them thoroughly? >> RUFER: So often when people talk about
encapsulation, they're really talking about taking a large part of the system and hiding
it behind a narrow interface. And what we were showing with the leading example is that's
not really encapsulation. When we talk about encapsulation, we have to talk about encapsulation
of one set of code or data with respect to other pieces of code and data. And if what's
happened is we've rolled off a large portion of the system behind a narrow interface and
there aren't seams between those individual pieces that we can use to maybe represent
it as classes that we can instantiate separately and exercise separately in tests. Then within
that large circle containing all that behavior, it's really not encapsulated. So it's a false
claim that that's true encapsulation because so much code is sitting in that area not encapsulated
from itself, multiple responsibilities that have access between themselves that they shouldn't
have. >> [INDISTINCT]
>> BIALIK: Private methods as well. >> [INDISTINCT]
>> BIALIK: Right. So the question is, the moment you mark a test private, now, there's
a question of testability. So what we would like to do is test our classes solely through
the public interface. And why is that? Because as soon as we mark something private, we're
saying the details of how this works is not important to the outside world. And we should
feel free to change private methods without regard to testability because we shouldn't
be testing those, right? We want to only drive our classes using tests as another client,
right? So we've got collaborating classes and we don't want to have two classes in the
production system basically knowing the details of the private structure. Similarly for a
test, we'd really like to only test through the public interface as much as possible.
Now, sometimes what happens is you end up with a class and it's got a private method
and it's doing something very complicated, right? And gosh, we'd really like to test
that. Sometimes what that ends up telling you is we should take that, pull that out
into a class and, you know, it's okay to have small classes and have that responsibility
to be something that we can test and use collaboration here to pull that piece out if it's really
important to test an it's sort of tucked away behind a private barrier.
>> RUFER: Yeah, in general, if you're feeling like you need to test something that's a private
method that almost always means that your class is doing multiple things. And one of
those things is that private method or, you know, a cluster of behavior underneath that
private method. A good class decomposition says isolate that behavior in its own class.
In this case, maybe we're talking about the gang of four strategy pattern.
>> Hi. This is a question regarding the country code example which are taken. So we basically
moved the country code out of which was designed as the proper hierarchy to a place where it
was being reference to. So is that a right solution or should we also look at implementing
a global kind of class or function which provides me that hierarchy access? Like the supposed--on
the user level I'm going to five hierarchies down, I can take that whole hierarchical access
and move it towards a global function which I could then use. That way, I not only solve
the problem of, in case I change a structure I change it only in one place, while also
sticking to that design which we had created. Because maybe some time down the line, the
same country code might now be required by some other class where we had initially designed
it for. >> RUFER: Right. So you're proposing a way
of trying to reduce the duplication of how we dig in and retrieve something like the
country code. But that doesn't solve all of our problems. For instance, Tracy discussed
the code that we're going to have to write in our tests just to create the structure
that eventually inside the test will be digging the country code out of. So we can add extra
code, extra complexity to reduce some of our problems but we can eliminate the problem
by inserting the country code directly into a class like our account. And, in fact, the
other flexibility problem that we discussed was we want to be able to look at a class-like
account and from the surface, from its public interfaces, say, "Ah, we know something important
about an account. It works with country code." If we just pass it, the user, even if now
we've got this function that makes it easier to retrieve the country code from the user,
we've still hidden what the actual interaction for account is.
>> Hello. Hey. >> RUFER: Hi.
>> So first of all I agree with 99.9% of what you were talking about.
>> RUFER: Ah. >> BIALIK: The point win.
>> RUFER: Point one. >> Which is a lot if--for people who know
me. You said there is one thing right in the beginning when you were talking about the
user preferences and you have made your argument by the multiple out that suddenly stopped
and then made a note. So this flexibility pulling out the preferences, isn't that encouraging
people to do speculative design and speculative coding? So you think a lot of what if there
are changes and what if that happens and what if that happens.
>> BIALIK: Right. >> RUFER: Okay.
>> But that might introduce unnecessary complexity into your design.
>> RUFER: I'm really glad you asked. Okay. >> It's a pleasure.
>> RUFER: So the term, speculative generality. And in fact, we're big fans of trying to suppress
speculative generality throughout our systems at Google. But there are two different kinds
of generalization. One is when you look out into the future and you say, "Maybe we should
implement some fancy specific mechanism for a change that we're concerned will occur."
And often that change never comes and you've written complex mechanism that you didn't
really need. Now, you're carrying the extra complexity in your code. It's bad for any
number of reasons. But what we're really talking about here are general design habits that
we'd like developers to have throughout their development. We--as engineers, we're--and
managers, we're tragically bad at predicting the future for some specific change, but we
can, at least in the companies that we've worked with, we can be pretty sure change
will come somewhere. And that means trying to build your systems with a general level
of flexibility so that when that unanticipated change comes, you'll be able to accommodate
it wherever it is. Now, there maybe some cases where you don't--one of these designs that
we suggested is better for flexibility. Maybe you don't implement that right away but you
have it mind for when the flexibility comes. And part of our--part of what we're trying
to describe here with this correspondence between flexibility and testability is that
when you're trying to influence developers for how they should build their system if
the person you're trying to reach cares about one and not the other, you want to be able
to apply arguments for both. And if you're trying to motivate a developer to go with
a design that's more testable and you can also help them see that it's going to assist
with flexibility, they're going to be--and if they're resistant to testability itself
as a reason to modify the design that maybe what it takes to get them over the hump and
use the design technique that's probably best for both.
>> Hi, Tracy and Russ. It's great, the talk. Thanks. One query I have is like when you
create a class it exposes its services interface, right? And so one of the assumptions, there
is the semantics of the parameters is constant. I mean what if--and this is a query from all,
from the test design perspective, that what if the semantics of that class interface is
changing and not the actual variables themselves? Because if I--if that changes might just case
kind of changes. And if I try to understand or learn more about the semantics, I'll be
going on and violating the--violating basically encapsulation which is the fundamental of
"Oops." >> BIALIK: So, let me see if I can rephrase
your question and see if this is what you're asking. So we took a bit of artistic license
with the sender interface and we said, "Gee, sending emails and sending text messages and
sending buzzes, well, will just send this message." Is that your question that if we--we
may not have the same kind of data that's actually needed for sending an email versus
sending a text message and how do we then have a common interface for doing so?
>> Right. I mean just to like the car analogy which you provided. So if you have these sparkplugs
and engines which make a car, and what if the sparkplugs and the engine, they do things
differently than they were doing earlier? So if that is changing without changing the
interface itself, my test case kind of changes and--from a business perspective. So how do
I capture that without knowing about the implementation of the class itself? Is there any way to incorporate
that? >> BIALIK: Oh, is that someone outside?
>> RUFER: Oh, that's someone outside. >> BIALIK: I'm sorry. I thought that was someone
here too trying to jump in. So, yeah, so if we had--so the other--the following question
is if we had sparkplugs and there's two different kind of sparkplugs and they needed to behave
differently. Yeah, we're kind of in a bad situation we really--if we're using inheritance,
we really need to have this unified interface and have that work for everything. And so
if we can't make them identical as much as possible while we're trying to do that, otherwise,
yeah, we're stuck with sort of this, "Well, now, we're going to have to have conditional
logic to say, 'Well, it's this one kind or this other kind.'"
>> RUFER: So in the sparkplug case, they have different behaviors but they probably have
the same contracts through which you use the behavior and that's what you can establish
in the interface really. You only need common method that unifies the two sibling types
of sparkplug. I think the other case with the senders is a little bit more interesting
in the question of, "If different kind of senders need different kind of information
to do their sending, how do they get it?" So there's a couple of different ways. One
is, and this isn't typically preferred, but you cold pass a collection of parameters to
each of the senders that was long enough that each of the senders had what it needed and
some received parameters that they just didn't use. Second option is the Director could pass--is
that the Director? >> BIALIK: Musical scorer.
>> RUFER: Musical scorer could pass an instance of itself in along with the message and allow
each of the senders to come back and get whatever extra piece of information they need. Third
option is the musical scorer could bundle up some generally useful information and supply
that along with the message and it really depends on, you know, situation by situation
which one of those is the one to go with. But in my experience, usually, you can't find
one of these ways to--if the utility, the behavior that varies really is similar, you
can usually find a way to provide a common interface.
>> BIALIK: Is that it? Are we at time? Okay. >> RUFER: Oh, okay. Ankit is telling me we're
out of time. But it's been great talking at GTAC today and thank you very much.