Simples Rules make creating Big Balls of Mud impossible.

It has always amazed me.

I don't build Big Balls of Mud. Yet they are so common in the code
other people write. For those who don't know what a BBoM is, read..
http://www.laputan.org/mud/

A BIG BALL OF MUD is haphazardly structured, sprawling, sloppy,
duct-tape and bailing wire, spaghetti code jungle. Weve all seen
them. These systems show unmistakable signs of unregulated growth,
and repeated, expedient repair. Information is shared promiscuously
among distant elements of the system, often to the point where
nearly all the important information becomes global or
duplicated. The overall structure of the system may never have been
well defined.

I have finally put my finger on why I don't, nay can't build such a mess.

You see, over the years I have become utterly convinced there is only
one way to build large scalable systems, and assumed that everyone saw
it the same way.

And that way is Client / Service architectures.

Alas, I see ample evidence in the code around me that too many people
haven't figured this out..

If I have _any_ two interacting modules, in _any_ language I _always_
explicitly elect one of them
* to be the Service,
* and the other the Client.

I then allow my client to have knowledge of the servers interface, but
explicitly forbid _any_ knowledge of the service's implementation.
(Encapsulation / Information Hiding)

I also write the server as a generic service, it does only one task
and does it well, but is absolutely ignorant of the client. Not so
much as a single upward reference to the client.

I aim for the "sweet spot" between single use and totally generic. I
implement only that functionality required by my current list of
client modules.

However I forbid any misfeature or wart in the service that would
render the service unreusable to any future, as yet unthought of
client.

If the two modules are symmetric peers, I declare both peers to be
services that are utterly ignorant of each other, and create a
supervisor client program to wire them together.

I also typically make the peers implement a common abstract
interface, and the supervisor and the peers would be manipulating
that.

In C/C++ like languages I work on a strict basis of each service
having exactly one header file that is it's public interface, and one
implementation file.

I never combine services in a header file, or in an implementation
file.

Any service that is too big to fit comfortably in a single interface
file or implementation file is clearly not a single responsibility
service and I decompose it into more than one service.

One service may be a client of another, so long (and this is critical)
as no cycles of dependency are formed.

So let me make explicit what I have seen other programmers do, that
I strongly disagree with ...

* Not designating either module in an interacting pair as the
service or the client.... or

* ...worse (shudder).. switching labels depending on which direction
the current data/event flow under discussion is.

* the service having explicit upward references to symbols in the client.

* having a cyclic chain of dependencies. (ie. Some service implemented at
some level by one of its clients.)

* the client has explicit references to symbols that are clearly part of
the particular implementation of the service, not the interface.


Question to the Group.....

Do you do what I have called "client / service" programming or the Other
Thing?

If you do The Other Thing, why and what do you call it?


Why do Client Service architectures work so well?

1) It's an internally self-consistent/closed system approach. You can you can
build/decompose client/server systems out of/into client/server chunks. ie. No
matter how large/small you grow/shrink, you can maintain this structure.
All the way from single functions to million SLOC monsters.

2) If you stick to these rules, you _never_ get a Big Ball of Mud.

3) Finding _any_ upward references immediately tells you that your have a
tangle forming, so don't do that.

3) You can build them up in layers, and unit test at every interface without
pulling in the whole Big Ball of Mud.

4) Design by Contract asserts and invariant checks form an
excellent bracing internal test framework.

5) You can create controlled mock services that allow you to "test from
inside".



--


John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : john.carter{}tait.co.nz
New Zealand

On Wed, 10 Jan 2007 17:09:58 +1300, John Carter
<john.carter{}tait.co.nz> wrote:


>And that way is Client / Service architectures.
>

I use the OSI 7 layer model as an architecture pattern. Service
layers (i'm talking OO service layers) generally go down around layer
2 or 3. The object layer us usually much higher up.

AndyW

Responding to Carter...

> And that way is Client / Service architectures.

If you have followed my posts here for that last couple of decades, you
will realize that my gut reaction is: NS, DT. B-)

The notion of a directed, acyclic graph is very common in approaches to
application partitioning and dependency management. The techniques have
been around for decades. [As an example, see the Application
Partitioning category of my blog, which just paraphrases a
methodological approach (Shlaer-Mellor) that dates from the early '80s.]

> Why do Client Service architectures work so well?
>
> 1) It's an internally self-consistent/closed system approach. You can you can
> build/decompose client/server systems out of/into client/server chunks. ie. No
> matter how large/small you grow/shrink, you can maintain this structure.
> All the way from single functions to million SLOC monsters.
>
> 2) If you stick to these rules, you _never_ get a Big Ball of Mud.

I think there is a bit more to it, though. One needs a systematic
approach to applying client/service notions. IOW, client/service sounds
good but the devil is in the details and one needs a good methodology
for applying it.


*************
There is nothing wrong with me that could
not be cured by a capful of Drano.

H. S. Lahman
hsl{}pathfindermda.com
Pathfinder Solutions
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
"Model-Based Translation: The Next Step in Agile Development". Email
info{}pathfindermda.com for your copy.
Pathfinder is hiring:
http://www.pathfindermda.com/about_us/careers_pos3.php.
(888)OOA-PATH

John Carter wrote:
> It has always amazed me.
>
> I don't build Big Balls of Mud. Yet they are so common in the code
> other people write. For those who don't know what a BBoM is, read..
> http://www.laputan.org/mud/
>
> A BIG BALL OF MUD is haphazardly structured, sprawling, sloppy,
> duct-tape and bailing wire, spaghetti code jungle. Weve all seen
> them. These systems show unmistakable signs of unregulated growth,
> and repeated, expedient repair. Information is shared promiscuously
> among distant elements of the system, often to the point where
> nearly all the important information becomes global or
> duplicated. The overall structure of the system may never have been
> well defined.
>
> I have finally put my finger on why I don't, nay can't build such a mess.
>
> You see, over the years I have become utterly convinced there is only
> one way to build large scalable systems, and assumed that everyone saw
> it the same way.
>
> And that way is Client / Service architectures.
>
> Alas, I see ample evidence in the code around me that too many people
> haven't figured this out..
>
> If I have _any_ two interacting modules, in _any_ language I _always_
> explicitly elect one of them
> * to be the Service,
> * and the other the Client.
>
> I then allow my client to have knowledge of the servers interface, but
> explicitly forbid _any_ knowledge of the service's implementation.
> (Encapsulation / Information Hiding)
>
> I also write the server as a generic service, it does only one task
> and does it well, but is absolutely ignorant of the client. Not so
> much as a single upward reference to the client.
>
> I aim for the "sweet spot" between single use and totally generic. I
> implement only that functionality required by my current list of
> client modules.
>
> However I forbid any misfeature or wart in the service that would
> render the service unreusable to any future, as yet unthought of
> client.
>
> If the two modules are symmetric peers, I declare both peers to be
> services that are utterly ignorant of each other, and create a
> supervisor client program to wire them together.
>
> I also typically make the peers implement a common abstract
> interface, and the supervisor and the peers would be manipulating
> that.
>
> In C/C++ like languages I work on a strict basis of each service
> having exactly one header file that is it's public interface, and one
> implementation file.
>
> I never combine services in a header file, or in an implementation
> file.
>
> Any service that is too big to fit comfortably in a single interface
> file or implementation file is clearly not a single responsibility
> service and I decompose it into more than one service.
>
> One service may be a client of another, so long (and this is critical)
> as no cycles of dependency are formed.
>
> So let me make explicit what I have seen other programmers do, that
> I strongly disagree with ...
>
> * Not designating either module in an interacting pair as the
> service or the client.... or
>
> * ...worse (shudder).. switching labels depending on which direction
> the current data/event flow under discussion is.
>
> * the service having explicit upward references to symbols in the client.
>
> * having a cyclic chain of dependencies. (ie. Some service implemented at
> some level by one of its clients.)
>
> * the client has explicit references to symbols that are clearly part of
> the particular implementation of the service, not the interface.
>
>
> Question to the Group.....
>
> Do you do what I have called "client / service" programming or the Other
> Thing?
>
> If you do The Other Thing, why and what do you call it?
>
>
> Why do Client Service architectures work so well?
>
> 1) It's an internally self-consistent/closed system approach. You can you can
> build/decompose client/server systems out of/into client/server chunks. ie. No
> matter how large/small you grow/shrink, you can maintain this structure.
> All the way from single functions to million SLOC monsters.
>
> 2) If you stick to these rules, you _never_ get a Big Ball of Mud.
>
> 3) Finding _any_ upward references immediately tells you that your have a
> tangle forming, so don't do that.
>
> 3) You can build them up in layers, and unit test at every interface without
> pulling in the whole Big Ball of Mud.
>
> 4) Design by Contract asserts and invariant checks form an
> excellent bracing internal test framework.
>
> 5) You can create controlled mock services that allow you to "test from
> inside".
>
>

This is more or less the beauty of procedural/relational programming.
You *don't* create one big fat application, but lots of small and
medium apps that talk to the database and communicate between each
other almost entirely thru the database. Each small app is like a small
town that uses the Nile (database) to communicate and trade with other
villages (apps). Sharable libraries of routines are available, but not
forced. They are "voluntary frameworks" or utilities and kept small and
simple so that they can be reworked for specific needs. Generic
one-size-fits-all marriage-or-death frameworks are a lost cause. The
"Big EXE" syndrome/thinking of OOP creates big balls of mud. OO biz
modeling is Pork City.

(It is important to design the schemas reasonably well. Good or at
least reasonable schemas are one of the few "musts" for this to work.)

It's what I've been sayin'

-T-

>
> --
>
>
> John Carter Phone : (64)(3) 358 6639

On Wed, 10 Jan 2007 18:49:19 +0000, H. S. Lahman wrote:

> NS, DT

New Style Delirium Tremens? Sorry, not even Google could translate that!
(I checked. They haven't got American to English yet. Not even in Beta :-)


> The notion of a directed, acyclic graph is very common in approaches to
> application partitioning and dependency management. The techniques have
> been around for decades.

I know, You know.

So why do people keep on building Big Balls of Mud?

And even if the devil is in the detail, the detail tells you exactly where
to cut to make the design cleaner. There is an upward reference here, here
and here. Go snip, snip, snip and it's all untangled.

I think we're going to need an awful big bottle of Drano to clear this
swamp.


--


John Carter Phone : (64)(3) 358 6639
Tait Electronics Fax : (64)(3) 359 4632
PO Box 1645 Christchurch Email : john.carter{}tait.co.nz
New Zealand

On 10 Jan 2007 11:22:48 -0800, "topmind" <topmind{}technologist.com>
wrote:

>
>John Carter wrote:
>> It has always amazed me.
>>
>
>>
>> Why do Client Service architectures work so well?
>>
>> 1) It's an internally self-consistent/closed system approach. You can you can
>> build/decompose client/server systems out of/into client/server chunks. ie. No
>> matter how large/small you grow/shrink, you can maintain this structure.
>> All the way from single functions to million SLOC monsters.
>>
>> 2) If you stick to these rules, you _never_ get a Big Ball of Mud.
>>
>> 3) Finding _any_ upward references immediately tells you that your have a
>> tangle forming, so don't do that.
>>
>> 3) You can build them up in layers, and unit test at every interface without
>> pulling in the whole Big Ball of Mud.
>>
>> 4) Design by Contract asserts and invariant checks form an
>> excellent bracing internal test framework.
>>
>> 5) You can create controlled mock services that allow you to "test from
>> inside".
>>
>>
>
>This is more or less the beauty of procedural/relational programming.
>You *don't* create one big fat application, but lots of small and
>medium apps that talk to the database and communicate between each
>other almost entirely thru the database. Each small app is like a small
>town that uses the Nile (database) to communicate and trade with other
>villages (apps). Sharable libraries of routines are available, but not
>forced. They are "voluntary frameworks" or utilities and kept small and
>simple so that they can be reworked for specific needs. Generic
>one-size-fits-all marriage-or-death frameworks are a lost cause. The
>"Big EXE" syndrome/thinking of OOP creates big balls of mud. OO biz
>modeling is Pork City.
>
>(It is important to design the schemas reasonably well. Good or at
>least reasonable schemas are one of the few "musts" for this to work.)


From my experience many inhouse systems are built this way. Little
programs that do simple things revolving around a large database that
contains everything.

I suspect the DBAs job is full time just figuring out what the
database actually does half the time

Many of the so called OO people often need to realise that not
everything needs to be objectified. Procedural is good enough in most
cases (and can be quite easily mixed with OO programs).

AndyW wrote:
> On 10 Jan 2007 11:22:48 -0800, "topmind" <topmind{}technologist.com>
> wrote:
>
> >
> >John Carter wrote:
> >> It has always amazed me.
> >>
> >
> >>
> >> Why do Client Service architectures work so well?
> >>
> >> 1) It's an internally self-consistent/closed system approach. You can you can
> >> build/decompose client/server systems out of/into client/server chunks. ie. No
> >> matter how large/small you grow/shrink, you can maintain this structure.
> >> All the way from single functions to million SLOC monsters.
> >>
> >> 2) If you stick to these rules, you _never_ get a Big Ball of Mud.
> >>
> >> 3) Finding _any_ upward references immediately tells you that your have a
> >> tangle forming, so don't do that.
> >>
> >> 3) You can build them up in layers, and unit test at every interface without
> >> pulling in the whole Big Ball of Mud.
> >>
> >> 4) Design by Contract asserts and invariant checks form an
> >> excellent bracing internal test framework.
> >>
> >> 5) You can create controlled mock services that allow you to "test from
> >> inside".
> >>
> >>
> >
> >This is more or less the beauty of procedural/relational programming.
> >You *don't* create one big fat application, but lots of small and
> >medium apps that talk to the database and communicate between each
> >other almost entirely thru the database. Each small app is like a small
> >town that uses the Nile (database) to communicate and trade with other
> >villages (apps). Sharable libraries of routines are available, but not
> >forced. They are "voluntary frameworks" or utilities and kept small and
> >simple so that they can be reworked for specific needs. Generic
> >one-size-fits-all marriage-or-death frameworks are a lost cause. The
> >"Big EXE" syndrome/thinking of OOP creates big balls of mud. OO biz
> >modeling is Pork City.
> >
> >(It is important to design the schemas reasonably well. Good or at
> >least reasonable schemas are one of the few "musts" for this to work.)
>
>
> From my experience many inhouse systems are built this way. Little
> programs that do simple things revolving around a large database that
> contains everything.
>
> I suspect the DBAs job is full time just figuring out what the
> database actually does half the time
>
> Many of the so called OO people often need to realise that not
> everything needs to be objectified. Procedural is good enough in most
> cases (and can be quite easily mixed with OO programs).

I am not sure what you are implying by "good enough". Many OO'ers say
that OO's benefits only or mostly come about under "large complex
applications". This may indeed be true. But if one can get a better net
result by splitting big applications into more small and medium ones,
then the need to manage one big app is simply just gone. OO is overkill
for smaller apps.

It is roughly like saying that nukes are better at killing 20,000
buffalo than guns. However, if you avoid the need to kill 20,000
buffalo, then you can stick with the simpler guns and not have
radioactive rivers etc.

Thus, technically the OO'ers may be right with regard to size. But, OO
may be the right tool for the wrong problem.

-T-

John Carter wrote:
> On Wed, 10 Jan 2007 18:49:19 +0000, H. S. Lahman wrote:
>
> > NS, DT
>
> New Style Delirium Tremens? Sorry, not even Google could translate that!
> (I checked. They haven't got American to English yet. Not even in Beta :-)
>
>
> > The notion of a directed, acyclic graph is very common in approaches to
> > application partitioning and dependency management. The techniques have
> > been around for decades.
>
> I know, You know.
>
> So why do people keep on building Big Balls of Mud?
>
> And even if the devil is in the detail, the detail tells you exactly where
> to cut to make the design cleaner. There is an upward reference here, here
> and here. Go snip, snip, snip and it's all untangled.
>
> I think we're going to need an awful big bottle of Drano to clear this
> swamp.

One should be careful to distinquish between links that are domain
related (biz rules) and links that are an artifact of software
development. If there is a link in the domain, then it makes sense it
should be in our software model also. It is tough to make a silk purse
out of a sow's ear, and inharently messy biz logic will probably result
in messy software because the model must contain what the domain needs
also. (You can try to clean up the domain, but often that is a
sociopolitical issue rather than a technical one over which most of us
have limited control over.)

But I do find that much biz logic is easier to manage in tables than in
code because tables are easier to sift, sort, and filter to study. Code
is not that flexible. It is hard to regroup stuff with one's eyeballs.
You may be a code speed reader, but I am not. I would rather let the
machine alter the view of info to fit my eyes or needs rather than the
other way around. Plus tablizing some of the rules allows
non-programmers to manage them. There is book called something like
Headrush Patterns (Headbang? Headbash? X-gen talk of somekind) that
hard-codes coffee products into the app code. The programmer should not
be managing specific store products nor their categories (except for
rare exceptions). That is dumb. Yet people praise that damned book. OO
culture has F'd up software development ideas left and right. It is
crap like that which motivates me to rant my arse off. If you are
hard-coding each product and classification systems into code, it WILL
be a big ball of mud.

OO defenders will say, "it is just an example". But there is rarely any
such disclaimer. They tolerate crap and then wonder why OO gets a bad
name.

-T-


>
>
> --
>
>
> John Carter Phone : (64)(3) 358 6639
> Tait Electronics Fax : (64)(3) 359 4632
> PO Box 1645 Christchurch Email : john.carter{}tait.co.nz
> New Zealand

-T-

On 10 Jan 2007 15:41:37 -0800, "topmind" <topmind{}technologist.com>
wrote:

>AndyW wrote:
>> On 10 Jan 2007 11:22:48 -0800, "topmind" <topmind{}technologist.com>
>> wrote:
>>
>> >
>> >John Carter wrote:
>> >> It has always amazed me.
>> >>
>> >
>> >>
>> >> Why do Client Service architectures work so well?
>> >>
>> >> 1) It's an internally self-consistent/closed system approach. You can you can
>> >> build/decompose client/server systems out of/into client/server chunks. ie. No
>> >> matter how large/small you grow/shrink, you can maintain this structure.
>> >> All the way from single functions to million SLOC monsters.
>> >>
>> >> 2) If you stick to these rules, you _never_ get a Big Ball of Mud.
>> >>
>> >> 3) Finding _any_ upward references immediately tells you that your have a
>> >> tangle forming, so don't do that.
>> >>
>> >> 3) You can build them up in layers, and unit test at every interface without
>> >> pulling in the whole Big Ball of Mud.
>> >>
>> >> 4) Design by Contract asserts and invariant checks form an
>> >> excellent bracing internal test framework.
>> >>
>> >> 5) You can create controlled mock services that allow you to "test from
>> >> inside".
>> >>
>> >>
>> >
>> >This is more or less the beauty of procedural/relational programming.
>> >You *don't* create one big fat application, but lots of small and
>> >medium apps that talk to the database and communicate between each
>> >other almost entirely thru the database. Each small app is like a small
>> >town that uses the Nile (database) to communicate and trade with other
>> >villages (apps). Sharable libraries of routines are available, but not
>> >forced. They are "voluntary frameworks" or utilities and kept small and
>> >simple so that they can be reworked for specific needs. Generic
>> >one-size-fits-all marriage-or-death frameworks are a lost cause. The
>> >"Big EXE" syndrome/thinking of OOP creates big balls of mud. OO biz
>> >modeling is Pork City.
>> >
>> >(It is important to design the schemas reasonably well. Good or at
>> >least reasonable schemas are one of the few "musts" for this to work.)
>>
>>
>> From my experience many inhouse systems are built this way. Little
>> programs that do simple things revolving around a large database that
>> contains everything.
>>
>> I suspect the DBAs job is full time just figuring out what the
>> database actually does half the time
>>
>> Many of the so called OO people often need to realise that not
>> everything needs to be objectified. Procedural is good enough in most
>> cases (and can be quite easily mixed with OO programs).
>
>I am not sure what you are implying by "good enough". Many OO'ers say
>that OO's benefits only or mostly come about under "large complex
>applications". This may indeed be true. But if one can get a better net
>result by splitting big applications into more small and medium ones,
>then the need to manage one big app is simply just gone. OO is overkill
>for smaller apps.
>
>It is roughly like saying that nukes are better at killing 20,000
>buffalo than guns. However, if you avoid the need to kill 20,000
>buffalo, then you can stick with the simpler guns and not have
>radioactive rivers etc.
>
>Thus, technically the OO'ers may be right with regard to size. But, OO
>may be the right tool for the wrong problem.
>
>-T-

Good enough = fit for purpose (ie. it does the job ok)

With my radical purism approach to OO I have a set of rules that
define what an object is and what it isnt. Everything that fits in
the not-an-object basket gets done using procdural code (C or C++
which I call C with classes when used this way).

Not-an-object code is generally things like services, mixins,
handlers, coms routines and non-object frameworks.

Stuff that goes in the 'an-object' basket must satisfy the contraints
of identity, state, action and tangibility all of the time. But...
the internals of an object may leverage the stuff in other layers
(usually the framework and service code)

This allows for the objects to focus on doing objecty stuff without
having to sweat the detail of how they do it.

I tend to use the OSI 7 layer approach to application design, with the
non-OO stuff being in the lower levels. Often, when creating an
application one doesnt need the upper levels of code (for example back
end server code as an example that is just processing sets of raw
data), so one ends up with a small non-OO programlet so to speak.

At the end of the day, its the type of program that determines if it
becomes OO or not (rather than the approach [OO or procedural] one
takes to developing the program which is the normal method).

Andy

"John Carter" <john.carter{}tait.co.nz> wrote in message
newsan.2007.01.10.04.09.49.664107{}tait.co.nz...
>
> It has always amazed me.
>
> I don't build Big Balls of Mud. Yet they are so common in the code
> other people write.

Is it really? I don't see it much any more. At least not since I joined an
employer that doesn't hire incompetent developers.
>
> I have finally put my finger on why I don't, nay can't build such a mess.

because you are a professional. BBoM code is usually written by very junior
people, incompetents, or hobbyists. The juniors may benefit from your
rules. The other two won't.

Your rules are interesting. From my standpoint, they are mostly helpful
recitations of good practices that have been preached for years. (See any
of the books by Steve McConnell). As a result, I thought about going
through your thread and adding snide remarks about 'old' advice, but decided
against it. I think you deserve better than that. You clearly meant well.

I would encourage you, though, to consider ways in which you, in your
environment, can improve the practice of development through structural
means.

Let me explain my logic:
You are seeing a lot of code that fits the BBoM pattern. I am not. I think
that may be because my employer doesn't hire morons, and when one happens to
get in anyway, they are quickly moved to a non-programming role or
"encouraged to seek opportunities elsewhere."

Therefore, I conclude that your employer is willing to hire "inexpensive"
development resources or is not set up to prevent their output from becoming
a legacy. For whatever reason this occurs, fixing it starts with
recognizing it.

Ways to address this:
Advocate to improve hiring practices so that fewer 'fools' get in to
software development
Advocate to improve annual review practices so that 'less capable'
programmers will move to non-coding positions.
Create or use Software factories created by a team of competent folks to
make advanced designs available to all
Advocate for required continuous training or retraining of internal staff
Advocate for the establishment of a cross-training program, so that smart
people are rewarded when they train each other.
Bring in SDLC process which requires formal design and code reviews (not
my first choice, but it helps).
Establish competitions and recognition for excellence in design and
coding
Impose quality reviews of externally written code before accepting it
into the portfolio.

Techniques like this will work to keep the mud from burying you. The mud
will exist. Let it exist somewhere else.

--
--- Nick Malik [Microsoft]
MCSD, CFPS, Certified Scrummaster
http://blogs.msdn.com/nickmalik

Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
I do not answer questions on behalf of my employer. I'm just a
programmer helping programmers.
--