Unit Testing and Test Cases

Daniel T. wrote:

> In article <44035117.A32AC86E{}det.ua.pt>,
> Miguel Oliveira e Silva <mos{}det.ua.pt> wrote:
>
>>Water Cooler v2 wrote:
>>
>>>Thanks for your diligent reply, Laurent. It has been very helpful and
>>>interesting.
>>>
>>>I think I follow you and believe that my understanding has been correct
>>>so far. What still intrigues me is:
>>>
>>>1. What then, are pre-conditions?
>>>
>>>My initial assessment showed they were conditions to test inside the
>>>function's implementation, before the function returned valid input. If
>>>they're not that, what are they?
>>
>>They are the client's part of the routine's contract.
>>
>>It is up to the client's to ensure a routine precondition
>>*before* a call is attempted (however, a wise supplier
>>should protect itself from erroneous preconditions, as
>>explained below).
>
> A nice write up except for this one problem. If the supplier is able to
> protect itself from erroneous inputs, then it can define proper outputs
> for those inputs, thus making them no longer erroneous.
>
> Someone else's example:
>
> double inverse( double x ) {
> return 1/x;
> }
>
> The precondition of course is that 'x != 0' however, thats an easy thing
> to check, and as such one can define a particular output for that
> condition:
>
> // returns NaN if x == 0
> double inverse( double x ) {
> return 1/x;
> }

Although you could specify it to return a silent NaN that merely delays
detecting a hard error and contaminates other possibly meaningful future
calculations that use this NaN result.

Generating a trap in most cases would be preferable since it usually
signifies a serious programming error when division by zero occurs.

Your improved definition potentially breaks some code that might be
expected to work algebraically since you can no longer satisfy the
invariant property for the function inverse for all valid inputs.

x == inverse( inverse(x) )

whereas 0 != inverse( inverse(0)) under your definition.

And at present inverse(Nan) is also undefined.

Returning a suitably signed infinity might just be defensible under some
circumstances if the calculation must proceed and cannot be aborted.

Regards,
Martin Brown

In article <du2530$jog$1{}newsg4.svr.pol.co.uk>,
Martin Brown <|||newspam|||{}nezumi.demon.co.uk> wrote:

> Daniel T. wrote:
>
> > In article <44035117.A32AC86E{}det.ua.pt>,
> > Miguel Oliveira e Silva <mos{}det.ua.pt> wrote:
> >
> >>Water Cooler v2 wrote:
> >>
> >>>Thanks for your diligent reply, Laurent. It has been very helpful and
> >>>interesting.
> >>>
> >>>I think I follow you and believe that my understanding has been correct
> >>>so far. What still intrigues me is:
> >>>
> >>>1. What then, are pre-conditions?
> >>>
> >>>My initial assessment showed they were conditions to test inside the
> >>>function's implementation, before the function returned valid input. If
> >>>they're not that, what are they?
> >>
> >>They are the client's part of the routine's contract.
> >>
> >>It is up to the client's to ensure a routine precondition
> >>*before* a call is attempted (however, a wise supplier
> >>should protect itself from erroneous preconditions, as
> >>explained below).
> >
> > A nice write up except for this one problem. If the supplier is able to
> > protect itself from erroneous inputs, then it can define proper outputs
> > for those inputs, thus making them no longer erroneous.
> >
> > Someone else's example:
> >
> > double inverse( double x ) {
> > return 1/x;
> > }
> >
> > The precondition of course is that 'x != 0' however, thats an easy thing
> > to check, and as such one can define a particular output for that
> > condition:
> >
> > // returns NaN if x == 0
> > double inverse( double x ) {
> > return 1/x;
> > }
>
> Although you could specify it to return a silent NaN that merely delays
> detecting a hard error and contaminates other possibly meaningful future
> calculations that use this NaN result.
>
> Generating a trap in most cases would be preferable since it usually
> signifies a serious programming error when division by zero occurs.

I have no problem with that, it's still a defined result.

> Your improved definition potentially breaks some code that might be
> expected to work algebraically since you can no longer satisfy the
> invariant property for the function inverse for all valid inputs.
>
> x == inverse( inverse(x) )
>
> whereas 0 != inverse( inverse(0)) under your definition.
>
> And at present inverse(Nan) is also undefined.

We'd have to define that as well I guess. "ensure inverse(NaN) == 0"


--
Magic depends on tradition and belief. It does not welcome observation,
nor does it profit by experiment. On the other hand, science is based
on experience; it is open to correction by observation and experiment.

On Tue, 28 Feb 2006 16:38:41 +0000, Miguel Oliveira e Silva wrote:

> "Daniel T." wrote:

> It seems to me, that you are incorrectly mixing the world
> of normal program behavior, and the world of exceptions
> and exceptional behavior.

A behavior is either correct or not. A program exposing incorrect behavior
is incorrect. [That correctness might be undecidable changes here nothing.]

> Exceptions exist as a
> mean to deal with (detectable) incorrect programs
> (in DbC it is as simply as this).

Exceptions are used for control flow in correct programs. Exceptions in
incorrect programs are presumably incorrect.

>> and as such one can define a particular output for that
>> condition:
>>
>> // returns NaN if x == 0
>> double inverse( double x ) {
>> return 1/x;
>> }
>
> You surely can, but you surely shouldn't (or else
> you will be doing defensive programming and not
> DbC).

Extended numeric sets are proven to be mathematically correct and deliver
far safer and efficient computations than ones defined on bound numeric
subsets. NaN is a *legal* value of IEEE float. It is a valid outcome of
division a finite number by exact zero. Better could be only full-scaled
interval arithmetic with infinity ideals.

>> Now that's a *real* precondition.
>
> What do you mean by "real"?

= used to determine correctness of the program.

> (Surely you are not stating that runnable
> preconditions are not real!)

That is the only possibility, as I have shown in our previous discussion. I
think there is no need to repeat it.

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

"Dmitry A. Kazakov" wrote:

> On Tue, 28 Feb 2006 16:38:41 +0000, Miguel Oliveira e Silva wrote:
>
> > "Daniel T." wrote:
>
> > It seems to me, that you are incorrectly mixing the world
> > of normal program behavior, and the world of exceptions
> > and exceptional behavior.
>
> A behavior is either correct or not. A program exposing incorrect behavior
> is incorrect. [That correctness might be undecidable changes here nothing.]

Agreed.

> > Exceptions exist as a
> > mean to deal with (detectable) incorrect programs
> > (in DbC it is as simply as this).
>
> Exceptions are used for control flow in correct programs.

Not in DbC. That would be an unacceptable abuse of exceptions.

A correct program should never raise exceptions.

In DbC, exceptions are used to deal with incorrect programs
(broken contracts).

> Exceptions in incorrect programs are presumably incorrect.

Nope. They *correctly* signal an *incorrect* program
(as in physical sciences when an experience whose results
contradicts what was expected from a theoretical law,
proves *correctly* that the law is *incorrect*).

> >> and as such one can define a particular output for that
> >> condition:
> >>
> >> // returns NaN if x == 0
> >> double inverse( double x ) {
> >> return 1/x;
> >> }
> >
> > You surely can, but you surely shouldn't (or else
> > you will be doing defensive programming and not
> > DbC).
>
> Extended numeric sets are proven to be mathematically correct and deliver
> far safer and efficient computations than ones defined on bound numeric
> subsets. NaN is a *legal* value of IEEE float.

(So what?)

I fail to see suitable uses of a NaN number ("Not-a-Number" number)
other than to express an error within the number representation,
but - by all means - be free to use that defensive approach to
inverse [inverse(0) = NaN].

Just don't forget that the inverse function (as all functions) are
is not an end in it selves, but a mean to reach somewhere.
I'll bet that in 99.99...99% if the times the client of inverse
is not expecting to divide by zero, neither such result makes
any sense to whatever calculation required the use of this
function.

The class/routine implementor is free to set up whatever
contract he wants. However, once defined, their clients are
bound to observe them. So if the inverse function defines
(as it should) the precondition of a non-zero argument,
then we are in the presence of an incorrect program
when x equals zero (regardless of the routine's
postcondition).

> It is a valid outcome of
> division a finite number by exact zero. Better could be only full-scaled

> interval arithmetic with infinity ideals.
>
> >> Now that's a *real* precondition.
> >
> > What do you mean by "real"?
>
> = used to determine correctness of the program.

What?

Are you saying that the non-zero x precondition
of inverse is not useful to determine the correctness
of a program?

> > (Surely you are not stating that runnable
> > preconditions are not real!)
>
> That is the only possibility, as I have shown in our previous discussion.

(I missed that demonstration.)

If that is your opinion, then don't call it DbC, or else you
will be confusing everyone who does not know what is
this methodology.

> I think there is no need to repeat it.

No it is not. It is only necessary a little effort to understand
what I'm saying, and what DbC is.
[B. Meyer, Object-Oriented Software Construction, 2ed,
pages 331-438]

> --
> Regards,
> Dmitry A. Kazakov
> http://www.dmitry-kazakov.de

Best regards,

-miguel

--
Miguel Oliveira e Silva

"Miguel Oliveira e Silva" <mos{}det.ua.pt> wrote in message
news:4404AB97.37B38F68{}det.ua.pt...
> "Dmitry A. Kazakov" wrote:
>
>> On Tue, 28 Feb 2006 16:38:41 +0000, Miguel Oliveira e Silva wrote:
>>
>> > "Daniel T." wrote:
>>
>> Exceptions are used for control flow in correct programs.
>
> Not in DbC. That would be an unacceptable abuse of exceptions.
>
> A correct program should never raise exceptions.

What if the contract specifically states that the behaviour of a given
method is to raise an exception? E.g.

/*
Pre-conditions: None.
Post-conditions: NullPointerException must be raised.
*/
void raiseNPE() {
/*Implementation goes here*/
}

>
> In DbC, exceptions are used to deal with incorrect programs
> (broken contracts).
>
>> Exceptions in incorrect programs are presumably incorrect.
>
> Nope. They *correctly* signal an *incorrect* program
> (as in physical sciences when an experience whose results
> contradicts what was expected from a theoretical law,
> proves *correctly* that the law is *incorrect*).

Not nescessarily, as shown above (i.e. the exception might correctly be
part of the behaviour of a correctly implemented program). However, if a
program is incorrectly implemented, then perhaps the exception raising
routines are also incorrectly implemented, so raising the exception might
not be a correct signal for anything!

Another example:

/*
Pre-condition: none.
Post-condition: "Hello World!" is printed to the standard out, no exception
will be raised.
*/
void printHelloWorld() {
print "Hello World!"
throw new DivideByZeroException();
}

This program is incorrectly implemented, and coincidentally an exception
is raised. But the exception being raised is *INCORRECTLY* signalling the
the incorrectness of the program; it claims that the problem had something
to do with a division by zero, when that was not the nature of the
incorrectness at all. Actually, the incorrectness was simply the presence of
the exception itself!

- Oliver

In article <4404AB97.37B38F68{}det.ua.pt>,
Miguel Oliveira e Silva <mos{}det.ua.pt> wrote:

> No it is not. It is only necessary a little effort to understand
> what I'm saying, and what DbC is.
> [B. Meyer, Object-Oriented Software Construction, 2ed,
> pages 331-438]

I understand DbC, but DbC is not what I've been discussing here.
Preconditions existed long before Mr. Meyer. Maybe it would avoid
confusion if you used 'require' instead.

As in, "a require clause will cause an exception if its condition is
false." This is very different from what happens if a precondition is
not met.

pre?con?di?tion: n : A condition that must exist or be established
before something can occur or be considered; a prerequisite.

If a precondition is not met, the behavior of the code cannot even be
considered...

With DbC we can wonder what would happen if I sent 0 to the inverse
function, because there is a defined result (an exception will be
generated.) Here, I've been talking about real preconditions, ones that
require being met or we cannot determine the outcome.

In article <aF2Nf.7647$Cp4.7169{}edtnps90>,
"Oliver Wong" <owong{}castortech.com> wrote:

> "Miguel Oliveira e Silva" <mos{}det.ua.pt> wrote in message
> news:4404AB97.37B38F68{}det.ua.pt...
> > "Dmitry A. Kazakov" wrote:
> >
> >> On Tue, 28 Feb 2006 16:38:41 +0000, Miguel Oliveira e Silva wrote:
> >>
> >> > "Daniel T." wrote:
> >>
> >> Exceptions are used for control flow in correct programs.
> >
> > Not in DbC. That would be an unacceptable abuse of exceptions.
> >
> > A correct program should never raise exceptions.
>
> What if the contract specifically states that the behaviour of a given
> method is to raise an exception?

Good catch! Exceptions are defined behaviors that correct programs use
to signal problems. Incorrect programs can do literally anything.


> > In DbC, exceptions are used to deal with incorrect programs
> > (broken contracts).
> >
> >> Exceptions in incorrect programs are presumably incorrect.
> >
> > Nope. They *correctly* signal an *incorrect* program
> > (as in physical sciences when an experience whose results
> > contradicts what was expected from a theoretical law,
> > proves *correctly* that the law is *incorrect*).
>
> Not nescessarily, as shown above (i.e. the exception might correctly be
> part of the behaviour of a correctly implemented program). However, if a
> program is incorrectly implemented, then perhaps the exception raising
> routines are also incorrectly implemented, so raising the exception might
> not be a correct signal for anything!

Not in DbC, and that is where the confusion lies. Miguel is talking
about DbC and Eiffel's require clause, Dmitry and I are talking about
preconditions.

> Another example:
>
> /*
> Pre-condition: none.
> Post-condition: "Hello World!" is printed to the standard out, no exception
> will be raised.
> */
> void printHelloWorld() {
> print "Hello World!"
> throw new DivideByZeroException();
> }
>
> This program is incorrectly implemented, and coincidentally an exception
> is raised. But the exception being raised is *INCORRECTLY* signalling the
> the incorrectness of the program; it claims that the problem had something
> to do with a division by zero, when that was not the nature of the
> incorrectness at all. Actually, the incorrectness was simply the presence of
> the exception itself!

Yes! And why on earth should we be passing control back to the client
(by raising an exception) when we already know that it contains
incorrect code, we know that because it gave us data that didn't meet
our pre-condition?


--
Magic depends on tradition and belief. It does not welcome observation,
nor does it profit by experiment. On the other hand, science is based
on experience; it is open to correction by observation and experiment.

On Tue, 28 Feb 2006 19:59:19 +0000, Miguel Oliveira e Silva wrote:

> "Dmitry A. Kazakov" wrote:
>
>> On Tue, 28 Feb 2006 16:38:41 +0000, Miguel Oliveira e Silva wrote:
>>
>>> Exceptions exist as a
>>> mean to deal with (detectable) incorrect programs
>>> (in DbC it is as simply as this).
>>
>> Exceptions are used for control flow in correct programs.
>
> Not in DbC. That would be an unacceptable abuse of exceptions.
>
> A correct program should never raise exceptions.

???

>> Exceptions in incorrect programs are presumably incorrect.
>
> Nope. They *correctly* signal an *incorrect* program
> (as in physical sciences when an experience whose results
> contradicts what was expected from a theoretical law,
> proves *correctly* that the law is *incorrect*).

Whom they signal to? To the incorrect program? What could an *incorrect*
program *correctly* do about anything? Why would you trust to what an
incorrect program says?

Program: "I am a liar."

Is it?

>> Extended numeric sets are proven to be mathematically correct and deliver
>> far safer and efficient computations than ones defined on bound numeric
>> subsets. NaN is a *legal* value of IEEE float.
>
> (So what?)
>
> I fail to see suitable uses of a NaN number ("Not-a-Number" number)
> other than to express an error within the number representation,

It is the same misunderstanding as above. NaN is not an error, it is not a
number. It is like i=sqrt(-1), which is also not a [real] number, but an
ideal from another set [of complex numbers.]

> but - by all means - be free to use that defensive approach to
> inverse [inverse(0) = NaN].

IEEE floats don't form a ring. So the theorem above does not hold. So what?

> Just don't forget that the inverse function (as all functions) are
> is not an end in it selves, but a mean to reach somewhere.
> I'll bet that in 99.99...99% if the times the client of inverse
> is not expecting to divide by zero, neither such result makes
> any sense to whatever calculation required the use of this
> function.

That does not influence program correctness. A nice thing about ideals,
like NaN, is that they don't leak. If an algorithm is correct in R without
ideals it stays in R with ideals. So it is 100% safe. You can continue with
NaN to the point where R is needed. Note, not as a precondition, but as a
type conversion. This conversion will raise an exception for NaN. That
would be a correct behavior.

>>>> Now that's a *real* precondition.
>>>
>>> What do you mean by "real"?
>>
>> = used to determine correctness of the program.
>
> What?
>
> Are you saying that the non-zero x precondition
> of inverse is not useful to determine the correctness
> of a program?

No. I claim that only one of the following two statements is true for a
given program:

1. A predicate is a "real" precondition <=> determines correctness => its
value is not used (handled) by the program

2. A predicate is not a precondition => its value can be handled.

>>> (Surely you are not stating that runnable
>>> preconditions are not real!)
>>
>> That is the only possibility, as I have shown in our previous discussion.
>
> (I missed that demonstration.)

pre: true
function Liar return Boolean is
begin
return not Correct (Liar);
end Liar;
post : Liar

If things determining the program correctness were allowed for evaluation
in the program, that would allow construction of the liar paradox.

So if a precondition determines correctness, then it cannot be checked (=it
is "real"). If it is checked [and the system is not self-contradictory],
then it does not determine the correctness => it is not a precondition.

> If that is your opinion, then don't call it DbC, or else you
> will be confusing everyone who does not know what is
> this methodology.

So in your opinion the "real" DbC is self-contradictory? Cool!

--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de

Oliver Wong wrote:

> "Miguel Oliveira e Silva" <mos{}det.ua.pt> wrote in message
> news:4404AB97.37B38F68{}det.ua.pt...
> > "Dmitry A. Kazakov" wrote:
> >
> >> On Tue, 28 Feb 2006 16:38:41 +0000, Miguel Oliveira e Silva wrote:
> >>
> >> > "Daniel T." wrote:
> >>
> >> Exceptions are used for control flow in correct programs.
> >
> > Not in DbC. That would be an unacceptable abuse of exceptions.
> >
> > A correct program should never raise exceptions.
>
> What if the contract specifically states that the behaviour of a given
> method is to raise an exception?

I understand what you Daniel and Dmitry are trying to
argue in this point. But as I several times attempted to call
up to your attention, DbC(tm) has a much simpler view of
contracts, normal program behavior and exceptions.
In DbC exceptions are simply the response of a program
to an incorrect behavior due to a broken contract.

There is no need to put exceptions in contracts, because
in DbC when a runnable assertions is false an exception
is required to be raised.

You may argue that exceptions can be used in normal
program control flow (in Eiffel its much harder to do that,
because, unlike Ada/C++/Java its exception mechanism
is tightly bound to DbC). However, I must insist that
such use is inappropriate (even in those languages).
Not only normal conditional instructions are much
more safe, expressive and efficient, but also it mixes
up two different programming worlds: the one bound
to the expected normal behavior of programs, and the
one used to get along with exceptional incorrect behavior.

Also using exceptions for normal program control,
is little more than the disguise use of the unstructured
goto instruction (no more single point of entry and
single point of exit for program components).

The structured property of programs is extremely
important because it is one that allows Hoare/Floyd
formal view of attaching meaning to programs through
preconditions and postconditions (which was one of the
most important earlier work in which Meyer based
his DbC methodology).

> E.g.
>
> /*
> Pre-conditions: None.
> Post-conditions: NullPointerException must be raised.
> */
> void raiseNPE() {
> /*Implementation goes here*/
> }
>
> >
> > In DbC, exceptions are used to deal with incorrect programs
> > (broken contracts).
> >
> >> Exceptions in incorrect programs are presumably incorrect.
> >
> > Nope. They *correctly* signal an *incorrect* program
> > (as in physical sciences when an experience whose results
> > contradicts what was expected from a theoretical law,
> > proves *correctly* that the law is *incorrect*).
>
> Not nescessarily, as shown above (i.e. the exception might correctly be
> part of the behaviour of a correctly implemented program).

I was referring to DbC(tm) in which, as I have mentioned several
times, things are much simpler and responsibilities are much clear.

> However, if a
> program is incorrectly implemented, then perhaps the exception raising
> routines are also incorrectly implemented, so raising the exception might
> not be a correct signal for anything!

If the exception is raised as a result of a false assertion, then
it correctly signals a broken contract, even if that contract
is not the intended one.

> Another example:
>
> /*
> Pre-condition: none.
> Post-condition: "Hello World!" is printed to the standard out, no exception
> will be raised.
> */
> void printHelloWorld() {
> print "Hello World!"
> throw new DivideByZeroException();
> }
>
> This program is incorrectly implemented, and coincidentally an exception
> is raised. But the exception being raised is *INCORRECTLY* signalling the
> the incorrectness of the program;

Of course the use of the "throw" instruction in programs can
incorrectly raise exceptions (there is no such instruction in Eiffel).
If you take a closer look to my message you'll see that I was
referring to exceptions raised as a result of false assertions
(situation in which they are always correctly raised, even
if the assertion was not intended).

> it claims that the problem had something
> to do with a division by zero, when that was not the nature of the
> incorrectness at all.

Indeed.

> Actually, the incorrectness was simply the presence of
> the exception itself!

Just because it was not bound to assertions (and DbC).

> - Oliver

Best regards,

-miguel

--
Miguel Oliveira e Silva

"Daniel T." wrote:

> In article <aF2Nf.7647$Cp4.7169{}edtnps90>,
> "Oliver Wong" <owong{}castortech.com> wrote:
>
> > "Miguel Oliveira e Silva" <mos{}det.ua.pt> wrote in message
> > news:4404AB97.37B38F68{}det.ua.pt...
> > > "Dmitry A. Kazakov" wrote:
> > >
> > >> On Tue, 28 Feb 2006 16:38:41 +0000, Miguel Oliveira e Silva wrote:
> > >>
> > >> > "Daniel T." wrote:
> > >>
> > >> Exceptions are used for control flow in correct programs.
> > >
> > > Not in DbC. That would be an unacceptable abuse of exceptions.
> > >
> > > A correct program should never raise exceptions.
> >
> > What if the contract specifically states that the behaviour of a given
> > method is to raise an exception?
>
> Good catch! Exceptions are defined behaviors that correct programs use
> to signal problems. Incorrect programs can do literally anything.
>
> > > In DbC, exceptions are used to deal with incorrect programs
> > > (broken contracts).
> > >
> > >> Exceptions in incorrect programs are presumably incorrect.
> > >
> > > Nope. They *correctly* signal an *incorrect* program
> > > (as in physical sciences when an experience whose results
> > > contradicts what was expected from a theoretical law,
> > > proves *correctly* that the law is *incorrect*).
> >
> > Not nescessarily, as shown above (i.e. the exception might correctly be
> > part of the behaviour of a correctly implemented program). However, if a
> > program is incorrectly implemented, then perhaps the exception raising
> > routines are also incorrectly implemented, so raising the exception might
> > not be a correct signal for anything!

> Not in DbC, and that is where the confusion lies.

All my messages clearly state that I'm writing about DbC(tm).

In fact, the first message I sent (In a thread named: "Design by
contract and unit tests") was a response to an incorrect behavior
attributed explicitly to DbC.

> Miguel is talking about DbC and Eiffel's require clause,

I'm not referring specifically to Eiffel's require clause.

DbC can (and should) be used in languages other
than Eiffel (I use it in all the languages I use and teach).
Of course it is be much more difficult to properly
use DbC in languages such as Ada/C++/Java.
For example, in Eiffel class assertions (invariants,
preconditions and postconditions) are part of the
class interface (not its implementation).

> Dmitry and I are talking about preconditions.

So am I, and how they relate to class contracts.
Of course I'm not only referring to Hoare/Floyd's
mathematical concept but to the practical DbC
preconditions.

If you take a closer look to my messages you'll see
we don't disagree that much regarding the theoretical
mathematical concept.

> (...)

Best regards,

-miguel

--
Miguel Oliveira e Silva