Python Design Patterns

MALE SPEAKER: I'd like to introduce author and Googler
Alex Martelli, and he will be telling us all about Python
design patterns.
Thank you very much.
[APPLAUSE]
ALEX MARTELLI: Thank you, everybody.
Actually, I'll be telling you about half or one third of it,
depending, because this is just the first part of a talk
that will need to be split into two or three parts,
depending on some issues.
So let's say that for every argument, we can roughly
classify how much we already understand about it, how much
study and practice we have, to level hardly ever heard about
it, haven't even heard about it, or just
barely, of level zero.

Apprentice is like, I've started, don't really know
much yet, but I'm certainly started,
journeyman, or master.
So how many of you would say about Python, I
hardly heard about it?

How many consider yourselves apprentices?
How many journeymen?
How many masters of Python?
Now, the same question about design patterns.
I've hardly heard about them?

Apprentice, just getting started?
OK.
Journeymen, mastered them pretty well?
Master, could teach about those things?
OK.
So we have just about the right level of audience.
I think we can split it in two by compressing about three
hours of materials into two hours' worth of presentation.
This is roughly how this is structured.
The Shu, Ha, Ri traditional levels more or less correspond
to apprentice, journeyman, master, although there other
implications there.
I aim for pretty low to medium level for design patterns, not
really giving any absolutely revolutionary insight there,
and somewhat higher than that about Python.
I don't really teach any of the basics.
I may give some insight.
But roughly, this is the number of slides I have on
each subject for a total--
I haven't really counted, it's about 90.
So we'll try to cover about half of it, that is basically
all the way to structural patterns this time, and in the
second part, focus on behavioral patterns.
Q&A, besides the end of the talk, are welcome anytime, but
of course, I may be just about to answer whatever question
you're thinking of in the next slide or so.
So please try to wait a second before asking, just to see if
I'm already going to answer it.
So patterns.
What's patterns?

Unless you're hardly ever heard about it, that the very
concept was born about 30 years ago in the words of
architect and philosopher Christopher Alexander, who was
basically thinking about buildings and towns, places
where people live and work.
They're made of entities he decided to call patterns.
And by looking at things in terms of languages of patterns
interacting with each other, the different details can be
subsumed in a grander view.
He also has this much more--
from his third book, "A Pattern Language," very
verbose but very important quote, "Every pattern
describes a problem which occurs often, and the core of
the solution to the problem in such a style that if we really
understand and apply it, we can apply the same solution
without ever doing things the same way twice." That is the
essence of patterns in the most general sense possible.
A crucial issue which Alexander stresses only 1,000
times in his books, and his followers maybe another 500 in
the software books, is that design is not independent of
technology.
Patterns are not a design concept in the sense of being
independent from the technology to be chosen in
implementation.
If you're going to build in bricks or wood or concrete,
some things remain the same, but many
important things change.
You don't exactly use the same patterns.
Similarity, to quote Gamma, et al from the foundational book
of "Design Patterns as a Software Discipline," "Point
of view affects our interpretation of what counts
as a pattern, and choice of programming language is
important because it changes the point of view." That's why
I titled this talk, "Design Patterns for Python," because
I'm specifically addressing this point of view in my talk.
More generally, design patents thrive particularly within
object-oriented development, for no particular reason.
They could apply just as well to assembly level programming
or functional programming, but I guess object-oriented is the
mainstream nowadays.
Mostly thanks to that wonderful book which I was
holding a second ago, and to that incredible sequence of
conferences and proceedings and online debate and
constructive criticism that is the Pattern Language of
Programming series/community.

They were so hyped, and they're such a nice name that
three or four years ago, I was starting to get really worried
that they'd be swept away as a fad or fashion, as so many
other good and not so good ideas in software have been,
because people are always looking for a silver bullet.
Well, not us.
We know there isn't one, but a lot of other people.
And fortunately, the concept has matured and I think we're
past the danger of being swept away by hype.
And I keep repeating, they're not independent from
programming languages.
So what are the traditional design
patterns applied to software?
Well, they're not something like data structures or
algorithms on the simple end.
So a linked list or a quick sort is not a design pattern.
They're not even domain-specific architectures
for entire subsystems, something
that's too big and abstract.
So for example, HTTP sessions built on top of cookies,
that's not a design pattern.
That's to specific of the field of building sessions on
top of non-session or reoriented protocol.
So somewhere in the middle, between the low level things
and the domain-specific, more complicated things, that's
where we have the patterns.
To quote the Gang of Four again, "Descriptions of
communicating objects and classes--" that's because we
are in the old domain, we would say communicating
functions and higher order functions if we were talking
about functional programming--
"customized to solve a general design problem in a particular
context." Sometimes, we're talking mostly about classes,
but far more often, we're talking about
how the objects relate.
And the purpose of the pattern is a very generic description
or category.
What is this pattern all about?
And you're supposed to do a very formal write-up of a
design pattern.
I'm focusing on two really important aspects.
The name of the pattern is absolutely crucial because if
you don't name things, they don't stay
well in your memory.
And the known uses, which I will be quoting at every step,
and also refer to as KUs, are absolutely crucial.
If you cannot quote at least three examples from disparate
pieces of software, it's not a design pattern.
That's absolutely crucial in both Alexander's conception,
the Gang's of Four.
You don't invent a design pattern.
You discover it in nature, so to speak.
You discover it in the state of the art of
practitioners in the field.
And then there's a sort of formalized
terminology, such as forces.
Which forces, meaning constraints and desires, this
pattern solves, and so on?
But the complete schema is hardly ever appropriate.
It depends on what audience.
If you want to go and present at PLoP, then by all means, do
a completely formal write-up.
But if you're talking about the typical, in front of the
whiteboard design discussion with colleagues, then
something more simple, a little bit more, of course,
than just the name and no uses, but say the results in
rational or context and problem,
are reasonably important.
But you don't need to use the whole, complex approaches.
So there's plenty of books which I would recommend,
depending on exactly what you're going to do.
Most importantly, I would not, repeat not, recommend the
original book to anybody except people at Ri level.
If you already know all there is to know about design
patterns, that is the time to read this book.
It's definitely not introductory.
I would not recommend "Pattern Hatching" to anybody except
those who have already read "Throughout Design Patterns,"
critiqued it, and are able to argue convincingly as to why
this is right but that was wrong.
Because this is basically what [? Vlissides ?] does here.
I would not recommend this one to anybody who's already
married, because it could put your marriage in danger, but
otherwise, it's quite a cool book, very
funny, as you can expect.
Seriously, the best introductory books are
essentially those two.
Shalloway and Trott for very formal, classroom-like, and
"Head First" for if you like thinking visually and with a
bit of disorder very [INAUDIBLE].
This is Bob Martin, extremely recommended, because the only
really good book that merges design patterns as in what
comes out of your design with methodologies, what goes into
your design, how do you proceed with that?
And a few others.
This is, of course, strictly if you are in a C++ shop where
you are actually allowed to use all of C++.
Otherwise, you'll feel very frustrated.
So as Alpert, Brown, and Woolf, in one of the books I
recommend, have already pointed out extremely clearly,
I'm just summarizing their arguments, many classic design
patterns are workarounds for the
existence of static typing.
So they wrote the "Smalltalk Companion." About half of the
Gang of Four were very strong Smalltalk programmers, but
they basically took their Smalltalk hats off when they
wrote the book and accepted static
typing as a way of life.
It need not be so.
It's not in Smalltalk, it's not in Python, it's not in
many other languages.
So basically, forget all the static typing constraints if
you're going to program in Smalltalk or Python.
And vice versa, there are specific strengths of the
languages you're using with, such as dynamic abilities,
introspection, and so on, which you may really want to
capitalize on.
And I keep focusing on names as important, which is why I
keep using the classic names, even when things appear very
different in Python.
Some patterns simply disappear because they get incorporated
into standard libraries and languages.
A typical example is iterator.
Modern languages have Iterator built in or in the standard
library, so the Iterator pattern, which covers quite an
important role in the original book, absolutely has no reason
to exist as a pattern, any more than you would consider
it a pattern.
Subroutine call.
If you are programming battery machine code, knowing there is
a pattern called subroutine call whereby you push the
return address somewhere before you jump to somewhere
else, it would be crucial, but your language offers coding to
subroutine as a built-in feature of the language, it's
not a pattern anymore.
Same goes for Iterator.
So the categories of purpose of the piece in the
traditional framing, which I'm strictly following, would be
creational, structural, behavioral.
The creational ones are the ones that are concerned with
instantiation of objects, the structural ones deal with
mutual composition of classes and objects.
You can imagine something that exists in a certain way.
And the behavioral ones deal with the interaction.
Of course, there's a few creational patterns, a bit
more structural ones, and an enormous sea
of behavioral ones.
Because the number of different important
interactions is much higher than the number of possible
structural relationships, which in turn swamps the
number of creational things.
So I hope to cover today some of the partners in these first
two, and leave the behavioral stuff for the second part.
Something that holds, no matter where, no matter what
language, no matter what else-- as long as you're doing
object-oriented design patterns at least--
two absolutely crucial ideas which pervade the book,
pervade the design pattern thinking--
program to an interface, not to an implementation.
This is also a great idea to do if you're doing machine
code, not just in object-oriented programming.
How careful you have to be to do that depends on your
language as well.
In Python, it's generally automatic.
Unless you take real trouble to do things wrong, you are
doing things right.
You are implicitly using duck typing, which is the extreme
form of programming to an interface.
In other languages, you have to be a bit more careful about
it, not all that much.
And favor object composition over class inheritance.

So when object-oriented programming was first
introduced, inheritance was basically seen as essentially
synonym with object-oriented programming.
You had to do everything by inheritance.
It took decades, I would say, to basically find out all the
ways in which this is not a good idea, and wean ourselves,
collectively speaking, from overuse of inheritance in
favor of more flexible mechanisms for composition.
I still remember the first time I read a book on C++
style which told me, it's wrong to
inherit car from engine.
Why?
It's so handy.
Yes, well, a car is not an engine.
A car has an engine, and so on and so forth.
That was over 20 years ago, but it's taken the industry
collectively almost as much.
But most of us still probably tend to inherit too match
unless we sit and think about it because inheritance is so
handy, and the costs are not obvious until you have
progressed a bit with your design.
So inherit only when it's convenient.

I know the party line is supposed to be, inherit only
when you can assert a list of invariants and "is A." Well, I
think you cannot ever assert, "is A." I think, as the
[? Kozinski ?]
put it, the "is" of identity is the most widespread error
in thinking and proceeding the world.
So you can never assert "is" if you want to be really
fanatic about it, at least not if you're into [? Kozinski, ?]
[? Litkinstein, ?]
and others of my favorite thinkers.
On the other hand, checking that it's really convenient.
Yes, you want to expose all methods that are in the base.
You want to do a lot of reuse, a little override, and maybe
extend a little bit, and you are perfectly fine with being
so tightly coupled with a cluster inheriting from that
you will never entangle the coupling.
Then if all of this holds, then it's right.
Otherwise, go for hold or wrap.
Hold and wrap, in the case of Python, are a subtle
distinction.
I insist on doing it like these kids are holding the
python, as you can see, which is much safer than having the
python wrapped around you.
But that depends on how big is the python.
This one is particularly large.
So by "hold," I mean an object has a certain subobject as an
attribute, and everybody, including the object itself
and external clients, know that, and
code O.S. some method.
It's simple, it's direct, it's immediate.
The problem is that the coupling is very strong, but
it's all wrong.
You're exposing the detail that O is implemented by
having an S, and what method S needs to have to O and sundry.
The wrapping is a hold by private name, or most often by
private name, and a delegation.
So everybody else calls O.method as opposed to
O.S.method.
You can do that explicitly, like in every language, by
defining method to call itself as method, or you can do it
automatically with getattr.
We'll see a few examples later because it's such a common
Python idiom.
The point is you get the coupling right.
Read up on the Law of Demeter.
It's not particularly Python-specific, but it's a
very super-simplified guideline to good
object-oriented design.
Never have more than one dot.
If you're saying, O.S.method, you have two dots, and you
know something's wrong.
Now, it is oversimplified, but it's a handy rule of thumb.
So this is an example of wrapping being much more
appropriate than inheritance because
inheritance cannot restrict.
If you inherit, you first of all, among other things,
commit to exposing every single method and public
attribute of your base class.
So what if you want to do all but a few?
Well, you can do that by wrapping to restrict.
So this is an initialization which takes something to wrap
and a set of names to block and records it.
And then whenever you try to get any attribute on this
object with a certain name, the name is among the blocked
ones, then an attribute error is raised.
You claim it's not there.
Otherwise, you delegate to the wrapped object.
Incidentally, if the wrapped object itself doesn't have
this attribute either, they will raise the attribute error
you just propagated.
That's perfectly fine.
So this isn't exactly a pattern, but it's a typical
example of idiomatic use of getattr in Python.
Creational patterns.
They don't really play all that much of a role in Python,
although-- yes?
MALE SPEAKER: Isn't that an example
of a decorator pattern?
ALEX MARTELLI: I have decorator later, and I
wouldn't really claim it is, because it's removing
something as opposed to adding something.
Decorator normally adds something.
This removes something.
So yeah, I guess it's very, very close to decorator, but I
dealt with it here because of the removal aspect.
So it's important to have thought a little bit about
creational patterns, even though they don't really play
that much of a role in Python.
I've always wondered, why don't they?
Why are they so crucial in other languages?
Well, I would guess it is mostly because, as we'll see,
the factory concept is essentially built in Python.
You don't have to do something different to call a generic
factory function versus instantiating a specific
class, so it comes very handy.
But being aware of it is not a bad idea anyway.
So I'm dealing with two subcategories
of creational patterns.
One is, we want just one instance to exist.
Note in this case, I start by stating the problem rather
than any solution because--
or problem or context or forces we're solving.
We only want one instance of something to exist.
Normally, in object-oriented programming, you make a class
and then instantiate it as many times as needed.
Sometimes you think, right or wrong, that you only want it
instantiated once, rarely exactly three times or
something like that.
So let's stick with once.
The solution for about 99% of the cases in Python is don't
use a class, use a module.
The main thing about class is being able to instantiate them
repeatedly.
Just use a module, and that module will have intrinsically
just one instance.
If that module gets imported from several places, as you
know, Python internally makes sure only one instance of the
module is used.
There are some problems with that.
Modules don't support subclassing, so you can't
really do any tweaks.
Modules don't support special methods, so you can't use, for
example, arithmetic operators.
You can't do plus, times, divide by, which are sometimes
nice to have.
The second approach is just make one instance.
You only want one instance, well, make one instance and
use it, that's it.
Why do you need to enforce the
existence of just one instance?
This works for another 99% of the cases not covered
by the first 99%.
The only real issue with it is that you have to commit to the
time when that instance is made, because all the rest of
your software needs to be able to know there is an instance
around and know what name to access it from or which means
to access it from.
And then there's a classic singleton pattern, which I
call Highlander for reasons I hope are obvious.
The problem with singleton is that despite the claims in the
Gang of Four book, it really doesn't work with subclassing
very well at all.

We'll see it when you come to the Python code [INAUDIBLE]
language.
And then there's monostate.
Monostate is the name originally given to it, I
think it was in the "C++ User Journal,"
but I called it Borg.
And the one problem with it is that Guido hates it for
reasons he's never really explained to me.
OK.
I'm sorry, I didn't connect.
So the "there can be only one" is, of course, the
"Highlander" reference in this thing, which is much better
than Singleton, that to any bridge player means a suit in
which I only have one card.
It's very handy for doing graphs, but it's nothing to do
with object-oriented programming.
So Python has this special method, New, which is
essentially the place where, theoretically, new instances
of the class get constructed, and this thing guarantees it
only constructs a new instance once and then
keeps returning it.
Why do I claim that subclassing is a problem?
Well, suppose there is class Foo getting errors from
Singleton, and class Bar getting errors from Foo.
Now I instantiate Foo, and then I instantiate Bar.
What now?
Do I still have one instance of Singleton?
Which class is it, or do I suddenly have two?
Depends.
In this case, I made it per class, but this means there's
more than one instance of Singleton.
There's more than one completed as joint object, on
which a test is instanced, will say,
yes, it is an instance.
Alternatively, you can forbid that.
You can have this fail, but then you are not supporting
subclassing.
There is really no way to support
subclassing well in Singleton.
That's one of the big reasons Singleton is totally bankrupt.
And although it's most popular design pattern, you can do
some quick Google trends to find out the names of
[INAUDIBLE] design pattern, it never works in any language.
Because the "is A" concept means you can
always subclass something.
There's no, at least in C++, no way to say, this class is
not subclassible further.
But then instances of the subclass are instances of the
mother class, and that kind of breaks everything.

Once again, the quote.

So Borg is basically, forget about instances.
Who cares about instances?
As long as they all share the same state, they may as well
be one, and you're very happy.
There's absolutely no case in which this gives any problem.
So in monostate, so you do that in C++ the original
version, you do that by forcing every method to be
static, every data member to be static.
That unfortunately doesn't really support inheritance
either, so Robert Martin has a very nice critique of
monostate in Singleton in an essay on his
site, Object Mentor.

But in Python, you have reasonable control over the
object state.
So basically, you can set the dictionary where the object
owes all of its states to the shared state of a class.
And so you can make as many objects as you want, they will
all be the same.
Except in terms of identity, in terms of everything they
will share the same state.
Any modification, any excess will always
go to the same state.
And when you want to subclass, you just have at one level of
the subclasses to assert with this, and of course, all
descendants until otherwise claimed have
the same shared state.
So basically, you're doing data overriding, which is a
typical Python feature.
It's not common to all object-oriented language,
whereby a class can override.
So you see, as long as you access it this way, the shared
state is not a static issue of this class Borg, but an issue
of which class is being generated.
This is the equivalent of what [INAUDIBLE]
calls a class method.
Python also has a class method.
So the other and much more interesting issue with
creational patterns is that we don't want to commit,
throughout our code, into
instantiating specific classes.
This is absolutely crucial, because this is where the
program to an interface and not to an implementation most
often breaks down in the real world.
People do strive hard to program to the interface, and
now they need a new widget.
And they do code, new widget, or I think it's new widget in
both Java and C++.
And there, in that very spot, they have destroyed their
independence from the act of implementation because they're
instantiating a specific class.
So this is why this whole group of creational patterns
is so important, because we absolutely need to uproot that
from your code.
There are essentially two approaches.
One, which I will not deal in-depth, but is very well
covered on [INAUDIBLE], is dependency injection, which
basically means your objects never create other objects.
They always get whatever they need
injected from the outside.
This is extremely wonderful for testing purposes, and it
does help a lot avoiding the specific class problem.
The problem with dependency injection, per se, it
doesn't cover you.
OK, you don't know how many widgets you need.
Part of your object's work is to determine that and generate
five, or seven, or 22 widgets, and you don't
really know how many.
How do you inject those dependents into the object in
the first place?
You don't.
You inject a factory.
There is a factory is something that can--
some callable, some function, for example--
that can create new objects of whatever class is
appropriate--
the factory knows--
or can reuse old ones, if needed.
They can be a method, overwritable by subclasses.
They can be functions for other callables.
And you can have an abstract factory class that basically
is a collection of methods for creation of factory methods
consistent with each other.
So in Python, each and every type and class is
essentially a factory.
Because you call it-- it's a callable like any other--
you call it like you would call anything else.
And this is the most crucial thing that makes it
interchangeable with any other callable, which could be a
function or whatever else.
Internally, it may implement that new special method we
mentioned, which means it can actually do anything.
It can do anything in terms of instantiation.
It doesn't actually have to give out new instances.
And it may be injected directly.
Since classes are objects, there's no need to have a
boilerplate function to code the class, just pass the class
as an argument, and there you are.
It can be injected.
Modules also can be kind of abstract
factory in a non-OO way.
For example, you never import POSIX or import NT.
I've never seen it in Python code anywhere.
You import OS.
And OS is the module which knows or finds out, OK, am I
on a POSIX operating system or an NT operating system, and
basically subsumes and generates a single instance of
one or the other as needed.
You cannot inherit and further refine the idea, which is why
classes are more flexible, but sometimes you
don't really want--
at least in design terms, we don't really want to have
different implementations of the operating system interface
module, depending on which variant of Windows NT we are
on, or which variant of Unix systems we're on.
Although it might be a better idea, this isn't the way it
was designed in the first place.
So this is the only known use I'm going to give for any
creational pattern in Python.
And it's not really in the library, which kind of
underscores the fact that it's not all that used.
Whenever you call a type intrinsically, it basically
starts with a new method.
And if the new method actually returns an instance of the
class it's been asked for, then that instance gets
initialized.
So Python intrinsically does a two-phase construction, which
is a pattern not in the book, but very popular in other
realms, particularly in GUIs you often have.
You don't really want to initialize the window object
as soon as it's created, because you first need to
establish, connect it in various and different ways,
and then all the
initialization must take place.
So two-phase construction is the constructor proper does
hardly anything, and then all the hard work is done in a
separate initialization method.
It's very popular in many realms, and Python basically
embodies it internally this way.
This an example of a very highly generically factory
function, one which is able to load any object from any
package and reach.
You give it the name of the package and the name of the
object, and it does everything else.
Of course, it would be easy to design it with a single
argument, which is the dot separated name, and does the
split on the last dot.
But since the role of the package name and the object
name are different, I decided.
This uses a built-in Import function, which basically
given a package, a couple of dictionaries to hold context,
and possibly a list of object names that will be needed,
returns the module from the package.
And then get attribute can take, in particular, the name
object from.
So for example, you can dynamically load-- of course,
this doesn't make any sense because it's absolutely
equivalent to from p1, p2, p3, import c4 as CLS.
So there's a language construct for that.
But by using this function, you can get these strings from
all sorts of places and do dark magic, which is probably
not the best idea to do in production code, but can be
very handy for testing, as usual.
So this just about covers my extremely short coverage of
creational patterns.
This could be a great idea for questions, because I'm not
going to talk about creational patterns every again.

OK, let's move on to structural patterns.
I'm actually only dealing with one subcategory in this series
of talks for now, which I call masquerading and adaptation.
I mostly do that because finding a nice side picture
for structure is almost impossible, while for mask,
you can find pretty cool ones.
This is my main motivation.
No really, these five buttons--
Adapter, Facade, Bridge, Decorator, and Proxy--
have something in common.
They're all about objects that basically take the place of
another, or make believe they are another.
OK, that's a bit vague, but let's go case by case.
So Adapter tweaks an interface at either
class or object level.
You need to consider both variants.
Facade simplifies a subsystem's interface.
Bridge allows you to build many implementations of an
abstraction using many different implementations of a
certain functionality without having to
repeat it in the coding.
Decorator reuses and tweaks without inheritance.
And Proxy decouples access to an object's functionality from
where the object is, whether you're allowed to reach
for it and so on.
So Adapter.
We have some client code gamma that requires a protocol C.
Protocol is my favorite generic
equivalent of interface.
I see interface as something that's purely syntactical,
while protocol implies pragmatic constraints as well.
Supplier code sigma provides some different protocol, S,
which basically has all that C needs, but arranged in
different ways.
So I write some adapter code that sneaks in the middle
between the client and the supplier.
It looks like a supplier to the client, it looks like a
client to the supplier, and implements protocol C by
making calls to protocol S.
So let's have a toy example.
Very, very toy.
What we require is a method foobar that
takes a foo and a bar.
Unfortunately, what we are given is a method, barfoo,
that takes a bar and a foo.
So how do we reduce the [INAUDIBLE]
mismatch with an adapter?
For example, let's stick to object-oriented.
We have a class whose method, barfoo, takes a bar and a foo.
And our client code is called to use a different library,
were the classes have a method foobar, which
wanted a foo and a bar.
This is how we do it with wrapping delegation.
So we write the wrapper, which takes the wrapee and saves it.
And whenever you call foobar with foo and bar, it delegates
that to barfoo on the wrapee with bar and foo.
It's not rocket science, but it's the tiniest and yet
useful example of adapter that I could put together, and this
is how you instantiate it.
You pass the existing barfooer as part of the instantiation.
If you want to work at a class level, you can do it with
inheritance.
So basically, you delegate to yourself.
So a foobar, in this case, inherits from a barfooer, and
adds method foobar, which calls barfoo.
This isn't necessarily appropriate, because you also
end up with the original barfoo.
So this is maybe faster, but the other approach, the per
object approach, is definitely cleaner.
There are many known uses of adapter in the
Python Standard Library.
Oh, I forgot to mention that all my Known Uses are from the
Python Standard Library.
It's a large library, you can find examples.
It's pretty representative of good Python code.
For example, a socket can be seen as a file object.
It has an underlying file object to adapter, which
basically wraps the role functionality of socket into
the rich interface of file objects.
That's the case of an adapter that actually has to do a lot
of work because sockets are unbuffered and files are
buffered, so all the buffering functionality is hidden there.
I don't think that's very appropriate.
I would not like to have a buffering class outside reuse
there, but that's how we coded it.
Don't know if you're familiar with the doctest, which are
very handy, fast, quick to write, but not quite as proper
and nice as the unit test.
But if you have a lot of doctests, you can adapt them
into unit tests with this testsuite adapter.
dbhash is another example of adapter.
Python Standard Library includes an interface through
the Berkeley Standard Distribution Database, which
is an extremely rich and complicated system.
Sometimes, we want to access it just if was a good old dbm
kind of file from the early days of [? Unix. ?]
dbhash gives us that.
String IO, if you have a string and want to access it
as if it was file, you don't have to wrap it in this, which
would be silly, you wrap it in a StringIO.
Shelve basically takes something that has a very
limited subset of dictionaries, essentially
forcing string keys and values and offering just the very
basic method, and make it look just like a rich dictionary
because it uses pickle, so it can translate any object to a
string and back, and uses UserDictMixin to add all the
other dictionary functionality.
So from these Known Uses and some other looking at real
world code, we see that in real life, some adapters
require a lot of code, a bit more than the foobar, barfoo
example I did, which is why I
identified it as a toy example.
Mixing classes are absolutely a great way to
adapt to rich protocols.
You can implement advanced method, nice, easy things.
I would exempt mixin classes from my general dislike of
inheritance.
I think mixin classes are the greatest thing
since sliced bread.
Adapters occur at all levels of complexity.
I can actually imagine, although I don't remember a
specific case, but I would code an adapter as simple as a
barfoo to foobar.
And you can't really get any simpler than that, and on the
other end of the spectrum, you really do use adapter for such
things as dbhash and bsddb.
And it's not just about classes and their instances.
In particular, we'll see that after facade, but callables
are a very good target for that.
So a facade occurs when some supplier code, a rich
subsystem, ideally, provides a lot of complexity, a lot of
complex and rich functionality, and we need a
simple subset.
Not necessarily because we've already written to that simple
subset, but just because we don't want a bazillion other
users of our code to have to code to this complicated thing
when they only need one tenth of the functionality.
So basically, the facade code implements and supplies the
simple subset C, by making internally its
calls to S as needed.
So what's the difference between a facade and adapter?
It's kind of subtle.
Adapter is about supplying a given protocol required by
client code.
So you imagine yourself already having a client, so
the protocol you need is fixed.
Or you are designing the client protocol to be
homogeneous so you can use polymorphism.
Facade, on the other hand, is about having this very rich
interface that does a bazillion things when you know
only a few is needed, and you want to present them in a very
simple to use way.
And most often, although this will not come very clearly
across from the Known Uses, because I haven't found any,
but in real life, facade typically fronts for many
objects, for a complex subsystem full of stuff, while
adapter is typically a one on one relationship.
So the best way to present facade, I found is in this
site, which is extremely controversial.
If you like controversy about design patterns and PHP and
MySQL, you will find them aplenty here.
This is the concept.
This is how it would be without facade, so I have my
subsystem with a lot of classes, and everybody from
the outside is needing to make calls to many classes.
With a facade, I take a little extra step, so basically
everybody calls the facade, and the facade deals with the
complexity of the subsystem.
So I don't use many figures here, apart from silly
ornamentation, but I really like this one because it shows
you the advantage.
Basically here, you're making the life of every client class
very complicated because they need to learn about all sorts
of important internal details of your subsystem.
And you're making your future life hell because you will be
constrained forever to keep this complex interface around
because everybody's making calls on your insides.
If you do go to the trouble of offering a facade, the lives
of the clients are simpler because all the things they
need are concentrated in one, simplified place.
And your life is simpler because the moment you decide
to redo this completely, you only need to make sure you can
keep emulating the facade, and your
maintenance is much easier.
This is far more than the simpler adapter provides.
Unfortunately, the Known Uses don't really support that.
I have not found any real, complicated
subsystem with a facade.
Email has a bit that.
The email package is pretty rich inside, but the reason
for the [INAUDIBLE] of the facade is actually to emulate
an older legacy implementation of email sending, so it's not
really the best of examples.
The facades I found are basically simplifying adapters
of single objects, which is not exactly the best idea.
For example, in [INAUDIBLE], there's a type fi.fo which
facades for list.
It just gives you push and pull functionality on top of
list rich functionality.
You can see dbhash as facading for bsddb.
This is probably the closest we get to a facade.
I also gave it as an example of an
adapter, so I'm cheating.
Old sets, now sets are built in, but they used to be a
facade on top of dicts.
Queues do a lot of stuff, but one way of framing them is a
facade for a double-ended queue with a lock, because
they're threat safe.
os.pash does some facades, too.
So in real life, facade may have
substantial amounts of code.
The important thing is to simplify the interface, even
if that means you do have to add a bit more code,
sometimes, minor functional additions like [INAUDIBLE]
method.
It occurs at all levels of complexity, but it really
matters when you're really dealing with complicated
subsystem, otherwise it's kind of gratuitous.
And inheritance is never useful.
Since you're simplifying, you're restricting, and since
you're restricting, inheritance is useless.
Inheritance always widens interfaces,
never restricts them.
And this is an example, both adapting and facading also
works for callables, not just in
object-oriented cases in Python.
Callables play a very large role in Python programming
because you can easily pass them as arguments, return them
as results, so you often need to adapt or facade them.
So there's functoolspartial, which performs currying
operations.
So the most elementary kind of simplification you can do on a
callable is take some of the arguments and functoolspartial
lets you do that.
And you have decorator syntax, they add something to make it
as easy as can be to apply a higher order function to wrap
an existing callable.
Closures, basically nested functions, provide with
extreme directness for the simple needs you may have in
terms of [INAUDIBLE], and of course, you can always write a
class with a special method call.
This, by the way, is from an economic fundamentals text.
It explains how to compute the worth of callables compared to
standard loans.
Callables means loans that can be repaid in
advance with no penalty.
So bridge.
You have a lot of realizations, rho, of a
certain abstraction, A. Each of them could be using any of
several implementations, you already have functionality F,
and you don't want to code N1 times N2, boilerplate classes.
So what you do is make sure the abstract superclass of all
your abstractions holds a reference to the abstract
super of interface of all the possible implementation.
And each rho has to go through R to reach F. That's the
easiest way I have to bridge.
This is a toy example in that I wouldn't do it this way.
But it's one way to do a bridge.
So let's say we have an abstract parser, and all the
abstract parser does is accept a scanner object and
hold it, and then--
sorry, I skipped.
You need two underscores here.
A typo on my part.
Every time it scans for any attribute, it asks the scanner
for that attribute.
So that is essentially the same idiom I was using
earlier, except it doesn't do any addition nor any
subtraction either, so it's even simpler.
But the point is that any parser can subclass abstract
parser, and whenever, say needs the next token or needs
to push back a token on a pushback stack, it doesn't
have to worry about what scanner approach.
As long as the interfaces are close, it basically codes them
in this case on self.
That's because I've used this idiom.
It could be self.
If I had [INAUDIBLE], I would say self.scanner.next_token,
self.scanner.pushback.

So basically, they all get funneled there.
So there's several Known Uses, such as socket servers, one of
the most complex.
And with a decorator, decorator crab, in particular,
which basically is how to insert some semantic tweak.
Basically, it's kind of like an adapter, except that the
incoming and outgoing interfaces are identical.
And this is kind of a complex subject for 10 minutes left.
I would point out the zip file, which basically looks
like a file and wraps a file, but inserts dynamically and
transparently compression and decompression in the middle.
Recursive locks, which look just like locks but know who
is holding them and allow the same thread who already holds
them to acquire them again.
Codecs, which basically transform a byte stream to a
byte stream, but with some typically internationalization
[INAUDIBLE].
And this is the [INAUDIBLE]
proxy, by the way, is for access restriction, object
[INAUDIBLE]
remotely or in persisted forms.

weakref.proxy has to do with life time issue, which is a
typical use for proxy.
We want to be able to access some object as long as it
exists, but not keep it alive just for our purposes, for
example, for caching.
That's why weakrefs exists.

Shelf, which I already gave as another
example, also does proxy.
idlelib.RemoteDebugger is a typical example.
It lets you insert debugging into a remote process,
essentially by proxying for it.
And this is Part One.
And we do have seven minutes left for questions and
answers, so I hope there are some.
Otherwise, I rushed through the last part unwarrantedly.

I was kind of hoping I could cut this in two parts rather
than three, which is why I inserted the pause here.

Any questions?
MALE SPEAKER: [INAUDIBLE]?
ALEX MARTELLI: Yes.
I gave the URL at the very start, but
let me do that again.

http://www.aleax .it/goo_pydp.pdf, and that is
just one big PDF.
It covers both a part we've done today, and the part we'll
do sometime in the future.
We haven't set a date yet.
Boris?
BORIS: Alex, you gave great examples
from the Python library.
Do you have any good examples from Google?
I'm sorry, I'm supposed to repeat the question.
So you gave great examples from the Python libraries.
Do you have any good examples from Google codes?
Actually, now that I think of it, maybe we
should leave this for--
ALEX MARTELLI: I do have excellent examples from third
party libraries and applications.

For example, one [INAUDIBLE] important class of design
patterns which I haven't covered at all are design
patterns for asynchronous communication.
They're covered very well for C++ in a couple of books by
Dr. Schmidt, and they're incorporated inside the Python
library in a very simplified way
known as asynchor asynchat.
There's a third party package called Twisted, which is
basically the best implementation I've ever seen
of those patterns, bar none.
That includes ACE, which is a great library,
but Twisted is better.
So there's plenty of third party libraries.
You could look at important applications.
I have never really studied there, but I'm pretty sure you
can find a lot of--
like take Sugar, the interface of the One Laptop Per Child.
[? It's all ?] in Python.
I'm pretty sure they're using a lot of design patterns in an
interesting way.
It would be a great thing to do a report on that.
Or big applications written in Python such as Mailman,
Chandler, Django, there's so many of them that are open
source so you don't have to have strained excess.
Other questions?

MALE SPEAKER: So you've talked a lot about Python-specific
design patterns.
I was curious as to whether you thought that--

excuse me, you talk more about general design patterns, and I
was curious as to whether you think there are any that
you've discussed today that are more relevant in Python
than in other languages.
I mean, you've talked a little about some that are less.
ALEX MARTELLI: I tried to focus my discussion only on
those aspects which are relevant to Python, yes.
MALE SPEAKER: Right, but all of the design patterns that
you've talked about, as far as I could tell,
could be applied to--
ALEX MARTELLI: They all come from the--
MALE SPEAKER: Right, exactly.
ALEX MARTELLI: In the second part, I will get into one
design pattern in extreme depth.
This part has been more of a survey.
The second part will be mostly about not just behavioral
patterns in general, but three of them--
template method, state, and strategy.
And I really get into the issue of how introspection and
dynamic approaches serve there.
So I'm not talking about patterns that
only apply to Python.
I'm not very interested in that.
Besides, there are languages which are basically equivalent
in power, such as Ruby.
It would be pretty weird to have something that only
applies to Python and not to Ruby or vice versa, because it
would have to be some weird, syntactic peculiarity.
Iterators are different, but the iterator pattern doesn't
also apply to either language because they both have
iterators built in in different ways--
Smalltalk-like in the case of Ruby, and much more similar to
the Gang of Four design pattern in the case of Python.
But in either case, they're built into the language, so
that's not so much a design pattern
anymore in those cases.

MALE SPEAKER: I had a question that I noticed in your talk
when you were working on one of the patterns where you were
wrapping an object.
And it occurred to me that you were wrapping the object in
such a way that someone that was looking into the object
wouldn't see those methods as part of the interface.
ALEX MARTELLI: You mean for introspection purposes?
MALE SPEAKER: Right.
And I was wondering, that's a danger that a language like
Python has that you never run into in C++.
And I wondered if you could comment on what we need to do
to consider that in Python.
ALEX MARTELLI: You could run it when doing C++ if you were
doing debugging with some advanced debugger, including a
class browser, so you get this object
which is actually proxying.
You may not actually be able to tell what it does have.
Or if you're using dynamic cast or advanced aspects of
C++, depending how smart your debugger is, it
may get rather confused.
If you specifically want to support interactive debugging,
you basically have to add all the introspection to your
wrapper to make sure it looks to the outside, including a
debugger, exactly as it would look in the inside.
It's a lot of overhead, unless you're specifically very
interested in interacting debugging.
As you know, I go by the motto, debugging sucks,
testing rocks.
So I'm much more interested in supporting extended testing
than in supporting extended debugging.
When I am debugging something, I'm going to be aware of the
actual issues anyway.
So for example, suppose I'm debugging something and I have
problems on a proxy that actually is supposed to send
the calls out Australia.

I'd better know that, because the reason things are
misbehaving may have nothing to do with my code and
everything to do with an underground cable through the
Pacific Ocean having just been cut by an earthquake.
So trying to help proxies and other [INAUDIBLE]
be totally transparent is possibly a misguided expense
of labor, given these issues, particularly in case of, OK
doesn't work, now what?
Now, is there are actually a transatlantic
cable in the middle?
You can't really abstract from that, because it could get cut
from an earthquake.
MALE SPEAKER: All right, I think we're out of time,
actually, so thanks very much.
ALEX MARTELLI: OK, thank you very much.
[APPLAUSE]