Re: Mixing P/R philosophy with OO (within J2EE).

On 10 Apr 2007 01:02:48 -0700, Harry wrote:

> A first-semester programmer, IMO, can very
> quickly, almost overnight, become an expert in identifying and writing
> procedures. Procedural is mainly action oriented (or, procedural). To
> carry out an action, there are steps... step 1, step 2, 3, 4... If
> there's a step 3.5 here and a step 2.23 there to be introduced, it is
> very easy and straightforward to do so.

I used to review code of such people, it is full of cut'n'paste. Students
don't understand abstraction, whether it be procedural, ADT or relational.
It takes much time and efforts to show them how to do it right, to analyse
the program, to be able to explain what the given piece of software does.

> If you need to add a new
> field to a C-struct, boom, you go ahead and add it.

In fact they don't use structures to bind related things together. It is
another abstraction most of them fail gasp.

You know, people don't have instinct of programming.

> OO is about modeling reality.

It is not.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Harry wrote:
> <snip>
> Thus, what I wanted to say was: in enterprise class systems of today,
> if the RDBMS component is very much in the picture, *and* if it is
> providing a nontrivial service with such solid mathematical
> foundations as set theory and predicate calculus, *why* would anybody
> want to ignore it and proceed to architect additional layer(s) on top
> of it?
>
That's a good point. A mythology exists that promotes the mapping of
object models to relational models--even at the expense of one, the
other, or both.
>
> <snip>
>
> My point of view, again, is that if the layer below (the RDBMS) is
> providing me with a very rich and valuable service, *and* if the
> procedural style is already available, why on earth would I want to
> write another set of abstraction layers if I'm perfectly happy and
> comfortable with the former's interface?
>
Actually, when you put it that way what does "procedural" have to do
with it? Aren't you advocating that relational databases are tools
programmers risk under-utilizing if they don't first try to exploit
everything an RDB may provide them?
>
>> <snip>
>
> Now that's a revelation to me! OO devoid of its somewhat
> anthropomorphic style of visualizing entities, I think, would leave
> big room for design errors. I'll never be absolutely sure if my
> concrete classes are correctly pointing to concrete or vague/abstract
> concepts.
>
Not devoid--just not "because." The goal isn't to create a Java version
of the real world, the goal is to create appropriately modeled Java
classes properly subdividing roles and responsibilities to maximize
flexibility, agility, expression, and maintainability. If the result
rhymes with the real world or if you find useful analogies there, that's
not a bad thing. It's just not the goal.


--
Visit <http://blogs.instreamfinancial.com/anything.php>
to read my rants on technology and the finance industry.

Dmitry A. Kazakov wrote:
> On 10 Apr 2007 01:02:48 -0700, Harry wrote:
>
> > A first-semester programmer, IMO, can very
> > quickly, almost overnight, become an expert in identifying and writing
> > procedures. Procedural is mainly action oriented (or, procedural). To
> > carry out an action, there are steps... step 1, step 2, 3, 4... If
> > there's a step 3.5 here and a step 2.23 there to be introduced, it is
> > very easy and straightforward to do so.
>
> I used to review code of such people, it is full of cut'n'paste. Students
> don't understand abstraction, whether it be procedural, ADT or relational.
> It takes much time and efforts to show them how to do it right, to analyse
> the program, to be able to explain what the given piece of software does.

That is true *regardless* of the paradigm. One has to walk before they
run. People need to get used to the basics of variables and loops
before they can focus their energies on big-picture design and long-
term maintenence. (Some suggest starting from the big-picture and
working their way down, but most students need to play with concrete
issues before playing with abstract ones. Of course, there are
exceptions.)

Factoring out duplication just because you can is not always the right
answer anyhow. You need experience to know why there is duplication in
the first place and the impact of factoring it out. Not all
duplication is bad, but only the hard knocks of experience will show
you why.

>
> > If you need to add a new
> > field to a C-struct, boom, you go ahead and add it.
>
> In fact they don't use structures to bind related things together. It is
> another abstraction most of them fail gasp.

Premature, perhaps.

>
> You know, people don't have instinct of programming.

I think many do as far as getting something up and running. I've seen
some pretty amazing things from people using merely Lotus 1-2-3
macros. As far as it being maintenance-friendly; well, that is why
they hire us instead.

> --
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de

-T-

"Harry" <simonsharry> writes:
> On Apr 10, 5:32 am, Patrick May <p...@spe.com> wrote:
>> >> > My current understanding is that the RDBMS is a layer [
>> >> > . . . ] providing me a very valuable, a very significant
>> >> > service in terms of something (set theory) that I'm well
>> >> > familar with since my pre-college days. In absence of any
>> >> > multi-vendor-RDBMS support requirement, I'd very much like my
>> >> > solution to be closest to this layer.
>>
>> >> Why?
>>
>> > Why?? What do you mean! If you had a good, very useful (let's
>> > say: an OO) layer (say, X) that was abstracting away something
>> > ugly (say, Y) and providing you a useful service, why would you
>> > want to build a few more layers on top of it and code against
>> > them instead of X.
>>
>> And how, exactly, does this demonstrate the value of
>> relational algebra to real world problem domains?
>
> Relational/procedural style predates OO. The burden of proving the
> inadequacy of relational/procedural in "real-world problem domains"
> lies on you, not me.

The burden of proof is on the person making the claim. You are
claiming that mapping the solution to an RDBMS and SQL provides value,
so it's your burden to prove it.

Attempting to shift the burden of proof is one of Bryce Jacob's
traits. Admit it: You're a sock puppet.

>> > The Facade pattern and its motivations don't apply to the above
>> > case because I (for one) am comfortable with the SQL interface,
>> > find it elegant enough, and would thus want to use, nay exploit,
>> > its full power.
>>
>> What power are you talking about, exactly?
>
> The ability to *very* easily relate/join, at runtime, two or more
> arbitrary tables (provided, of course, it makes sense) is what I
> believe is the single biggest powerful feature of SQL/relational.

How, exactly, does this translate into value in real world
problem domains (aside from those with an explicit need to join
arbitrary tables)? I'm trying to understand why you believe that all
problem domains are best addressed by mapping the solution to the
relational model.

>> To give you an idea of where I'm coming from, the systems I
>> build have behavior. They read data feeds, run Monte Carlo
>> simulations, compute risk, and feed the results to trading systems.
>> They provision telco network elements. They manage workflows that
>> integrate disparate COTS systems running over WANs. They broadcast
>> video via satellite. I want to implement these systems in a
>> language that matches the problem domain, not in SQL.
>
> And you truly believe P/R can't do any of those things?

No, and I never made that claim.

> Each of the names you've dropped above can be handled outside of OO
> just as elegantly, provided you're unbiased and open enough to see
> elegance in things other than OO.

I agree. Personally, I've found value in the ease with which
domain specific languages can be created in Common Lisp. I'm trying
to understand why you think that RDBMSs and SQL should be used
preferentially. How does being based on relational algebra provide
value in practice?

>> Of course not. You have, however, quoted two completely
>> bogus, unfounded assertions from Bryce Jacobs' site. First, that
>> OO is a rehash of the network database model. Second, that OO is
>> unsuitable for dynamic scenarios. No one with any experience with
>> OO techniques would make either claim.
>
> There's something in OO 'filosofy' itself (I think!) that tends to
> slow you down by encouraging (forcing?) you think hard /
> 'consciously' about issues which could otherwise be pragmatically
> ignored or which wouldn't normally surface or could be handled in
> alternate ways in the procedural world.

Can you provide specific examples?

> I'm not saying we all become doofuses and not exercise our gray
> matter, or that procedural does not require any thinking at
> all... but that, IMO, it does not typically involve that much of
> head-scratching. A first-semester programmer, IMO, can very
> quickly, almost overnight, become an expert in identifying and
> writing procedures. Procedural is mainly action oriented (or,
> procedural). To carry out an action, there are steps... step 1, step
> 2, 3, 4... If there's a step 3.5 here and a step 2.23 there to be
> introduced, it is very easy and straightforward to do so. If you
> need to add a new field to a C-struct, boom, you go ahead and add
> it.

And then you have to recompile and relink every module that uses
it. Then you have to retest everything you just changed.

> In OO however, you must also pay attention to nontrivial design
> aspects such as types, their roles, and their relationships.

You have to pay attention to detail to produce quality software
using any set of techniques. OO gives you more tools in your virtual
toolbox. That's a good thing.

> All this requires an insight which simply cannot be gained
> overnight; infact, in most cases it will take years and years of
> exposure to not only 'good' but also 'not-so-good' and 'bad'
> real-world OO designs.

Another issue that is technique independent. Just because
procedural coding may be easier for many students to learn initially
doesn't mean that it simplifies the creation of solutions to real
world problems. In fact, "beginner friendly" languages can be a
problem. As I tell my Java wielding colleagues, not entirely
facetiously, C++ is a far better language than Java because it is more
difficult to use. With C++ you need a certain level of understanding
in order to get to the point of having a badly running, incorrect
program. With Java, any idiot can get something running, and many do.

> And, if an effort *is* made to 'really look around' in the
> system, in the problem space... for commonalities, then it becomes a
> question of how much thorough / serious / sincere / pedantic /
> filosofical you want to get. For example, should I go for pure
> interfaces or abstract classes? Traits/aspects? Should I follow the
> pimpl idiom thruout my system? Should I forgo the use of new
> operator in favor of the Factory pattern? To how much extent should
> I go in making my classes suitable for extension/derivation? Is my
> class final enough to be designated as such (in Java)? If no, what
> can I do to make it so? Should I even worry about the finality of my
> classes? Should I go for MI (which C++ supports and also
> exemplifies via the iostream hierarchy), or should I not go for MI
> (for, if a successful OO language such as Java and the bright OO
> folks at Sun chose to not even support the feature, may be I too
> should stay away from MI (within C++))? What should be my Vehical
> hierarchy? Should TwoWheeler and FourWheeler be derived from
> Vehicle? Or, should Vehicle support a method 'int
> getNumberOfWheels()'? Or both? When introducing a Task type (that is
> capable of running in parallel with the primary thread of the
> system)... now, IS Task A Thread? Or, does Task HAVE A Thread within
> it? Should I derive it off of a new GenericTask (an abstract class
> or an interface)? Hmm... my system can in near future have
> sequential (non-parallel) tasks as well, so let me give this thing
> some more thought...! If I'm introducing GenericTask as an
> afterthought... is my existing class hierarchy robust enough to
> handle it... or is it fragile (let me double check this...)?
> Etc. Etc. Etc.

Developing quality software requires skill and experience. News
at 11.

> OO is about modeling reality.

No, it is not. The best short summary of the purpose of OO that
I've heard is that "OO is about dependency management." I'm not sure
I buy that 100%, but it's a good first (and second) approximation to
reality.

Sincerely,

Patrick

------------------------------------------------------------------------
S P Engineering, Inc. | Large scale, mission-critical, distributed OO
| systems design and implementation.
pjm@spe.com | (C++, Java, Common Lisp, Jini, middleware, SOA)

On Apr 11, 8:18 am, Patrick May <p...@spe.com> wrote:
> "Harry" <simonsha...> writes:
> > On Apr 10, 5:32 am, Patrick May <p...@spe.com> wrote:
> >> >> > My current understanding is that the RDBMS is a layer [
> >> >> > . . . ] providing me a very valuable, a very significant
> >> >> > service in terms of something (set theory) that I'm well
> >> >> > familar with since my pre-college days. In absence of any
> >> >> > multi-vendor-RDBMS support requirement, I'd very much like my
> >> >> > solution to be closest to this layer.
>
> >> >> Why?
>
> >> > Why?? What do you mean! If you had a good, very useful (let's
> >> > say: an OO) layer (say, X) that was abstracting away something
> >> > ugly (say, Y) and providing you a useful service, why would you
> >> > want to build a few more layers on top of it and code against
> >> > them instead of X.
>
> >> And how, exactly, does this demonstrate the value of
> >> relational algebra to real world problem domains?
>
> > Relational/procedural style predates OO. The burden of proving the
> > inadequacy of relational/procedural in "real-world problem domains"
> > lies on you, not me.
>
> The burden of proof is on the person making the claim. You are
> claiming that mapping the solution to an RDBMS and SQL provides value,
> so it's your burden to prove it.

Nonsense! You can safely assume,
1. I'm an OO dummy.
2. I'm comfortable with SQL/Relational and C-style functions.
Now, *you* need to come up with a sound argument, a convincing
example... of something that *cannot* be done in P/R that would force
me/others to look to OO as our savior! For an OO expert such as
yourselves dealing with 'Large scale, mission-critical, distributed OO
systems design and implementation', this should be a piece of cake.
Man, you can't be so abysmally pathetic! I bet you got badly sucked
into OO without having had the chance to learn and apply P/R
principles properly.

Suggestion: Stop resorting to endless issue evading and name
calling! I don't somehow believe you can be at SP Engineering and
also be *that* stupid so as not to understand with whom lies the
burden of proof in light of the above givens.

> Attempting to shift the burden of proof is one of Bryce Jacob's
> traits. Admit it: You're a sock puppet.

You have an unhealthy obsession with Bryce Jacobs. Is it because he
kicks your sorry a** all the time? Are you sure you're not
<somebody>'s c**dom puppet?

On Apr 11, 12:22 am, Thomas Gagne <tga...@wide-open-west.com> wrote:
> Harry wrote:
> > My point of view, again, is that if the layer below (the RDBMS) is
> > providing me with a very rich and valuable service, *and* if the
> > procedural style is already available, why on earth would I want to
> > write another set of abstraction layers if I'm perfectly happy and
> > comfortable with the former's interface?
>
> Actually, when you put it that way what does "procedural" have to do
> with it? Aren't you advocating that relational databases are tools
> programmers risk under-utilizing if they don't first try to exploit
> everything an RDB may provide them?

> Aren't you advocating that relational databases are tools
> programmers risk under-utilizing if they don't first try to exploit
> everything an RDB may provide them?

Yes. For a change, I'd like to think critically (something I haven't
been doing all these years), and I'd like to start bottom-up... at the
RDBMS layer.

> Actually, when you put it that way what does "procedural" have to do
> with it?

What I'm saying is: the RDBMS component is already an un-ignorable
given in all enterprise-class systems. Now, given also that I happen
to be comfortable writing loosely hanging, C-style functions/
procedures... what advantages OO and its fixed class hierarchies
approach is going to provide me over relational/procedural in terms of
time and money savings, and even elegance?

The answer that I've sort of gotten so far (to my original post) is:
if it's a social, cultural, political, career issue: go with the Java
and the general OO herd. An answer I've come to terms with already.
However, at the technical and purely merit level, I've not so far seen
anybody furnish any convincing argument for NOT sticking to the tried
and tested, simple and intuitive, and mathematically sound and
relatively less subjective P/R techniques.

"Harry" <simonsharry> writes:
> On Apr 11, 8:18 am, Patrick May <p...@spe.com> wrote:
>> The burden of proof is on the person making the claim. You
>> are claiming that mapping the solution to an RDBMS and SQL provides
>> value, so it's your burden to prove it.
>
> Nonsense!

No, my statement is an accurate summary of the burden of proof.
In this case, it lies squarely on you.

> You can safely assume,
> 1. I'm an OO dummy.
> 2. I'm comfortable with SQL/Relational and C-style functions.
> Now, *you* need to come up with a sound argument, a convincing
> example... of something that *cannot* be done in P/R that would force
> me/others to look to OO as our savior!

I have never, in this thread or any other, portrayed OO
techniques as a silver bullet. You, on the other hand, have made the
positive claim that mapping solutions to the relational model and SQL
provides value in practice. Defend your claim or retract it.

>> Attempting to shift the burden of proof is one of Bryce
>> Jacob's traits. Admit it: You're a sock puppet.
>
> You have an unhealthy obsession with Bryce Jacobs.

That's just what Bryce Jacobs would say, sock puppet.

Sincerely,

Patrick

------------------------------------------------------------------------
S P Engineering, Inc. | Large scale, mission-critical, distributed OO
| systems design and implementation.
pjm@spe.com | (C++, Java, Common Lisp, Jini, middleware, SOA)

"frebe" <frebe73> writes:
>> > The Facade pattern and its motivations don't apply to the above
>> > case because I (for one) am comfortable with the SQL interface,
>> > find it elegant enough, and would thus want to use, nay exploit,
>> > its full power.
>>
>> What power are you talking about, exactly?
>
> I think he is talking about the power of predicate logic and set
> theory.

How, exactly, does that translate to practical benefits in real
problem domains?

>> To give you an idea of where I'm coming from, the systems I
>> build have behavior. They read data feeds,
>
> Relational algebra is used for data management, it is not used for
> communication (or presentation).

Exactly my point, relational algebra doesn't add value in this
part of the problem domain.

>> run Monte Carlo simulations,
>
> Doesn't your Monte Carlo simulatons operate on data? Isn't relations
> a suitable way of organizing data in this case? Wouldn't relational
> algebra be helpful for doing parts of the calculations?

Not necessarily. The amount of data is huge. There isn't a
commercially available RDBMS that can handle the load in the time
available.

Now, that's not saying that a distributed, in-memory relational
database couldn't do so, but it's not clear what value relational
algebra would provide over other techniques in this domain.

>> compute risk,
>
> Yet again, relational algebra would probably be a very helpful tools
> for computing risk.

And again, there are no commercially available RDBMSs that can
perform acceptably in this problem domain. Consider a system
receiving a thousand ticks per second of market data and computing
risk on hundreds of portfolios with tens of thousands of positions
each. This information needs to be made available to other systems
twenty times a second.

>> I want to implement these systems in a language that matches the
>> problem domain, not in SQL.
>
> If the problem domain is data management, SQL matches it.

Not always. Mr. Kazakov has described the "nearest neighbor"
problem several times in this newsgroup. I don't recall seeing a
relational solution.

Another example I've described once or twice before comes from a
financial system similar to those I mentioned above. We had a
directed graph of transactions that had be settled in the order
specified by the edges. That is, a given transaction could not settle
until all of the transactions it pointed to had settled or it had to
settle at the same time as those transactions. The solution involved
identifying all cycles in the graph, converting them to single,
complex transaction nodes, and then doing settlement over the
resulting directed acyclic graph. This system is continuously
modifying the graph based on new input and settles a couple of million
transactions per day.

What value does relational algebra provide in these situations?

Sincerely,

Patrick

------------------------------------------------------------------------
S P Engineering, Inc. | Large scale, mission-critical, distributed OO
| systems design and implementation.
pjm@spe.com | (C++, Java, Common Lisp, Jini, middleware, SOA)

Now, per your suggestion, if I were to *retract* all that I've said so
far... even since the beginning of time(!), will you consider doing
the following?

> > You can safely assume,
> > 1. I'm an OO dummy.
> > 2. I'm comfortable with SQL/Relational and C-style functions.
> > Now, *you* need to come up with a sound argument, a convincing
> > example... of something that *cannot* be done in P/R that would force
> > me/others to look to OO as our savior!

Will you? Or, will you prefer remaining in your I-said-You-said
infinite loop?


> >> Attempting to shift the burden of proof is one of Bryce
> >> Jacob's traits. Admit it: You're a sock puppet.
>
> > You have an unhealthy obsession with Bryce Jacobs.
>
> That's just what Bryce Jacobs would say, sock puppet.

Er, let him say that himself. You probably don't realize this: you
accuse me of doing X (which I don't do), and then go ahead and do X
yourself anyways. In addition to X, you innocently think you don't do
Y.

Legend:
X = sock puppeting to BJ.
Y = big time sock puppeting to OO counterparts of BJ

On 10 Apr 2007 12:58:20 -0700, topmind wrote:

> That is true *regardless* of the paradigm. One has to walk before they
> run.

You have an idea of locomotion first.

> People need to get used to the basics of variables and loops
> before they can focus their energies on big-picture design and long-
> term maintenence. (Some suggest starting from the big-picture and
> working their way down, but most students need to play with concrete
> issues before playing with abstract ones. Of course, there are
> exceptions.)

No you don't gills, because your ancestors were fishes.

IMO it is a widespread delusion that training should follow the way science
took to research the thing being trained. In schools kids first learn the
physics of XVIII century, to hear 2-3 years later, forget it, that was
wrong. Same with mathematics, even worse.

P.S. I was lucky to participate in an experimental school program. We
started algebra just with the set theory, and used mathematical notation
right from the beginning. I don't remember any problems with that.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de